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Chapter 1 


Introduction 


1.1 Themes’ 


From its beginnings in the late nineteenth century, electrical engineering has blossomed from focusing on 
electrical circuits for power, telegraphy and telephony to focusing on a much broader range of disciplines. 
However, the underlying themes are relevant today: Power creation and transmission and information 
have been the underlying themes of electrical engineering for a century and a half. This course concentrates 
on the latter theme: the representation, manipulation, transmission, and reception of information 
by electrical means. This course describes what information is, how engineers quantify information, and 
how electrical signals represent information. 

Information can take a variety of forms. When you speak to a friend, your thoughts are translated by 
your brain into motor commands that cause various vocal tract components-the jaw, the tongue, the lips—to 
move in a coordinated fashion. Information arises in your thoughts and is represented by speech, which must 
have a well defined, broadly known structure so that someone else can understand what you say. Utterances 
convey information in sound pressure waves, which propagate to your friend’s ear. There, sound energy is 
converted back to neural activity, and, if what you say makes sense, she understands what you say. Your 
words could have been recorded on a compact disc (CD), mailed to your friend and listened to by her on her 
stereo. Information can take the form of a text file you type into your word processor. You might send the 
file via e-mail to a friend, who reads it and understands it. From an information theoretic viewpoint, all of 
these scenarios are equivalent, although the forms of the information representation—sound waves, plastic 
and computer files—are very different. 

Engineers, who don’t care about information content, categorize information into two different forms: 
analog and digital. Analog information is continuous valued; examples are audio and video. Digital 
information is discrete valued; examples are text (like what you are reading now) and DNA sequences. 

The conversion of information-bearing signals from one energy form into another is known as energy 
conversion or transduction. All conversion systems are inefficient since some input energy is lost as heat, 
but this loss does not necessarily mean that the conveyed information is lost. Conceptually we could use any 
form of energy to represent information, but electric signals are uniquely well-suited for information repre- 
sentation, transmission (signals can be broadcast from antennas or sent through wires), and manipulation 
(circuits can be built to reduce noise and computers can be used to modify information). Thus, we will be 
concerned with how to 


represent all forms of information with electrical signals, 

encode information as voltages, currents, and electromagnetic waves, 

manipulate information-bearing electric signals with circuits and computers, and 

receive electric signals and convert the information expressed by electric signals back into a useful 
form. 


1This content is available online at http: //cnx.org/content/m0000/2.18/. 
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Telegraphy represents the earliest electrical information system, and it dates from 1837. At that time, 
electrical science was largely empirical, and only those with experience and intuition could develop telegraph 
systems. Electrical science came of age when James Clerk Maxwell? proclaimed in 1864 a set of equations 
that he claimed governed all electrical phenomena. These equations predicted that light was an electro- 
magnetic wave, and that energy could propagate. Because of the complexity of Maxwell’s presentation, the 
development of the telephone in 1876 was due largely to empirical work. Once Heinrich Hertz confirmed 
Maxwell’s prediction of what we now call radio waves in about 1882, Maxwell’s equations were simplified 
by Oliver Heaviside and others, and were widely read. This understanding of fundamentals led to a quick 
succession of inventions—the wireless telegraph (1899), the vacuum tube (1905), and radio broadcasting—that 
marked the true emergence of the communications age. 

During the first part of the twentieth century, circuit theory and electromagnetic theory were all an 
electrical engineer needed to know to be qualified and produce first-rate designs. Consequently, circuit theory 
served as the foundation and the framework of all of electrical engineering education. At mid-century, three 
“inventions” changed the ground rules. These were the first public demonstration of the first electronic 
computer (1946), the invention of the transistor (1947), and the publication of A Mathematical Theory 
of Communication by Claude Shannon (1948). Although conceived separately, these creations gave birth 
to the information age, in which digital and analog communication systems interact and compete for design 
preferences. About twenty years later, the laser was invented, which opened even more design possibilities. 
Thus, the primary focus shifted from how to build communication systems (the circuit theory era) to what 
communications systems were intended to accomplish. Only once the intended system is specified can an 
implementation be selected. Today’s electrical engineer must be mindful of the system’s ultimate goal, 
and understand the tradeoffs between digital and analog alternatives, and between hardware and software 
configurations in designing information systems. 


NOTE: Thanks to the translation efforts of Rice University’s Disability Support Services?, this 
collection is now available in a Braille-printable version. Please click here* to download a .zip file 
containing all the necessary .dxb and image files. 


1.2 Signals Represent Information’ 


Whether analog or digital, information is represented by the fundamental quantity in electrical engineering: 
the signal. Stated in mathematical terms, a signal is merely a function. Analog signals are continuous- 
valued; digital signals are discrete-valued. The independent variable of the signal could be time (speech, for 
example), space (images), or the integers (denoting the sequencing of letters and numbers in the football 
score). 


1.2.1 Analog Signals 


Analog signals are usually signals defined over continuous independent variable(s). Speech, as 
described in Section 4.10, is produced by your vocal cords exciting acoustic resonances in your vocal tract. 
The result is pressure waves propagating in the air, and the speech signal thus corresponds to a function 
having independent variables of space and time and a value corresponding to air pressure: s (x,t) (Here we 
use vector notation x to denote spatial coordinates). When you record someone talking, you are evaluating 
the speech signal at a particular spatial location, x9 say. An example of the resulting waveform s (xo, t) is 
shown in Figure 1.1. 

Photographs are static, and are continuous-valued signals defined over space. Black-and-white images 
have only one value at each point in space, which amounts to its optical reflection properties. In Figure 1.2, 
an image is shown, demonstrating that it (and all other images as well) are functions of two independent 
spatial variables. 


*http://www-groups.dcs.st-andrews.ac.uk/~history/Biographies/Maxwell.html 
Shttp://www.dss.rice.edu/ 
“nttp://cnx.org/content/m0000/latest/FundElecEngBraille.zip 

>This content is available online at http: //cnx.org/content/m0001/2.27/. 
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Figure 1.1: A speech signal’s amplitude relates to tiny air pressure variations. Shown is a recording of 
the vowel “e” (as in “speech”). 


Figure 1.2: On the left is the classic Lena image, which is used ubiquitously as a test image. It contains 
straight and curved lines, complicated texture, and a face. On the right is a perspective display of the 
Lena image as a signal: a function of two spatial variables. The colors merely help show what signal 
values are about the same size. In this image, signal values range between 0 and 255; why is that? 


Color images have values that express how reflectivity depends on the optical spectrum. Painters long ago 
found that mixing together combinations of the so-called primary colors—red, yellow and blue-can produce 
very realistic color images. Thus, images today are usually thought of as having three values at every point 
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00 | nul |) 01 | soh|) 02 | stx |) 03 | etx || 04 | eot || 05 | enq]| 06 | ack)| O07 | bel 
08 | bs || 09 | ht OA | nl OB | vt OC | np || OD | cr OE | so || OF | si 
10 | dle || 11 | del || 12 | dce2|| 13 | dce3}] 14 | dce4]) 15 | nak]/ 16 | syn|| 17 | etb 
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60 |’ 61 ja 62 | b 63 | ¢c 64 | d 65 | e 66 | f 67 | g 
68 | h 69 | i 6A | j 6B | k 6C | 1 6D | m 6E | n 6F | o 
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Table 1.1: The ASCII translation table shows how standard keyboard characters are represented 
by integers. In pairs of columns, this table displays first the so-called 7-bit code (how many 
characters in a seven-bit code?), then the character the number represents. The numeric codes 
are represented in hexadecimal (base-16) notation. Mnemonic characters correspond to control 
characters, some of which may be familiar (like er for carriage return) and some not (bel means 
a “bell”). 


in space, but a different set of colors is used: How much of red, green and blue is present. Mathematically, 
; , ; T 
color pictures are multivalued—vector-valued-signals: s (x) = (r (x) ,g(x),6(x))°. 
Interesting cases abound where the analog signal depends not on a continuous variable, such as time, but 
on a discrete variable. For example, temperature readings taken every hour have continuous—analog—values, 
but the signal’s independent variable is (essentially) the integers. 


1.2.2 Digital Signals 


The word “digital” means discrete-valued and implies the signal depends on the integers rather than a 
continuous variable. Digital information includes numbers and symbols (characters typed on the keyboard, 
for example). Computers rely on the digital representation of information to manipulate and transform 
information. Symbols do not have a numeric value, however each is typically represented by a unique 
number but performing arithmetic with these representations makes no sense. The ASCII character code 
shown in Table 1.1 has the upper- and lowercase characters, the numbers, punctuation marks, and various 
other symbols represented by a seven-bit integer. For example, the ASCII code represents the letter a as 
the number 97, the letter A with 65. 


1.3 Structure of Communication Systems’ 


The fundamental model of communications is portrayed in Figure 1.3 (Fundamental model of communi- 
cation). In this fundamental model, each message-bearing signal, exemplified by s(t), is analog and is a 
function of time. A system operates on zero, one, or several signals to produce more signals or to simply 
absorb them (Figure 1.4). In electrical engineering, we represent a system as a box, receiving input signals 
(usually coming from the left) and producing from them new output signals. This graphical representation 
is known as a block diagram. We denote input signals by lines having arrows pointing into the box, 
output signals by arrows pointing away. As typified by the communications model, how information flows, 
how it is corrupted and manipulated, and how it is ultimately received is summarized by interconnecting 
block diagrams: The outputs of one or more systems serve as the inputs to others. 


®This content is available online at http: //cnx.org/content/m0002/2.17/. 
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Figure 1.3: The Fundamental Model of Communication. 


x(t) y(t) 
———> System }|——> 


Figure 1.4: A system operates on its input signal x (¢) to produce an output y (t). 


In the communications model, the source produces a signal that will be absorbed by the sink. Examples 
of time-domain signals produced by a source are music, speech, and characters typed on a keyboard. Signals 
can also be functions of two variables—an image is a signal that depends on two spatial variables—or more— 
television pictures (video signals) are functions of two spatial variables and time. Thus, information sources 
produce signals. In physical systems, each signal corresponds to an electrical voltage or current. 
To be able to design systems, we must understand electrical science and technology. However, we first need 
to understand the big picture to appreciate the context in which the electrical engineer works. 

In communication systems, messages—signals produced by sources—must be recast for transmission. 
The block diagram has the message s(t) passing through a block labeled transmitter that produces the 
signal x(t). In the case of a radio transmitter, it accepts an input audio signal and produces a signal that 
physically is an electromagnetic wave radiated by an antenna and propagating as Maxwell’s equations predict. 
In the case of a computer network, typed characters are encapsulated in packets, attached with a destination 
address, and launched into the Internet. From the communication systems “big picture” perspective, the 
same block diagram applies although the systems can be very different. In any case, the transmitter should 
not operate in such a way that the message s(t) cannot be recovered from z(t). In the mathematical sense, 
the inverse system must exist, else the communication system cannot be considered reliable. (It is ridiculous 
to transmit a signal in such a way that no one can recover the original. However, clever systems exist that 
transmit signals so that only the “in crowd” can recover them. Such cryptographic systems underlie secret 
communications. ) 

Transmitted signals next pass through the next stage, the evil channel. Nothing good happens to a 
signal in a channel: It can become corrupted by noise, distorted, and attenuated among many possibilities. 
The channel cannot be escaped (the real world is cruel), and transmitter design and receiver design focus 
on how best to jointly fend off the channel’s effects on signals. The channel is another system in our block 
diagram, and produces r (t), the signal received by the receiver. If the channel were benign (good luck 
finding such a channel in the real world), the receiver would serve as the inverse system to the transmitter, 
and yield the message with no distortion. However, because of the channel, the receiver must do its best to 
produce a received message & (t) that resembles s (t) as much as possible. Shannon’ showed in his 1948 paper 
that reliable—for the moment, take this word to mean error-free—digital communication was possible over 
arbitrarily noisy channels. It is this result that modern communications systems exploit, and why many 
communications systems are going “digital.” The module on Chapter 6, titled Information Communication, 
details Shannon’s theory of information, and there we learn of Shannon’s result and how to use it. 

Finally, the received message is passed to the information sink that somehow makes use of the message. 


"http: //www-gap.dcs.st-and.ac.uk/~history/Biographies/Shannon. html 
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In the communications model, the source is a system having no input but producing an output; a sink has 
an input and no output. 

Understanding signal generation and how systems work amounts to understanding signals, the nature 
of the information they represent, how information is transformed between analog and digital forms, and 
how information can be processed by systems operating on information-bearing signals. This understanding 
demands two different fields of knowledge. One is electrical science: How are signals represented and ma- 
nipulated electrically? The second is signal science: What is the structure of signals, no matter what their 
source, what is their information content, and what capabilities does this structure force upon communication 
systems? 


1.4 The Fundamental Signal: The Sinusoid* 


The most ubiquitous and important signal in electrical engineering is the sinusoid. 


Sine Definition 
s(t) =Acos(27ft+¢) or Acos(wt+@) (1.1) 


A is known as the sinusoid’s amplitude, and determines the sinusoid’s size. The amplitude conveys the 
sinusoid’s physical units (volts, lumens, etc). The frequency f has units of Hz (Hertz) or s~', and determines 
how rapidly the sinusoid oscillates per unit time. The temporal variable t always has units of seconds, and 
thus the frequency determines how many oscillations/second the sinusoid has. AM radio stations have carrier 
frequencies of about 1 MHz (one mega-hertz or 10° Hz), while FM stations have carrier frequencies of about 
100 MHz. Frequency can also be expressed by the symbol w, which has units of radians/second. Clearly, 
w = 2nrf. In communications, we most often express frequency in Hertz. Finally, ¢ is the phase, and 
determines the sine wave’s behavior at the origin (t = 0). It has units of radians, but we can express it in 
degrees, realizing that in computations we must convert from degrees to radians. Note that if 6 = —5, the 
sinusoid corresponds to a sine function, having a zero value at the origin. 


Asin (2r ft + ¢) = Acos (2nft+o-5) (1.2) 


Thus, the only difference between a sine and cosine signal is the phase; we term either a sinusoid. 

We can also define a discrete-time variant of the sinusoid: Acos(27fn+¢). Here, the independent 
variable is n and represents the integers. Frequency now has no dimensions, and takes on values between 0 
and 1. 


Exercise 1.1 (Solution on p. 9.) 
Show that cos (27 fn) = cos (27 (f + 1)n), which means that a sinusoid having a frequency larger 
than one corresponds to a sinusoid having a frequency less than one. 


NOTE: Notice that we shall call either sinusoid an analog signal. Only when the discrete-time 
signal takes on a finite set of values can it be considered a digital signal. 


Exercise 1.2 (Solution on p. 9.) 
Can you think of a simple signal that has a finite number of values but is defined in continuous 
time? Such a signal is also an analog signal. 


1.4.2 Communicating Information with Signals 


The basic idea of communication engineering is to use a signal’s parameters to represent either real numbers or 
other signals. The technical term is to modulate the carrier signal’s parameters to transmit information 
from one place to another. To explore the notion of modulation, we can send a real number (today’s 
temperature, for example) by changing a sinusoid’s amplitude accordingly. If we wanted to send the daily 


8This content is available online at http: //cnx.org/content/m0003/2.15/. 
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temperature, we would keep the frequency constant (so the receiver would know what to expect) and change 
the amplitude at midnight. We could relate temperature to amplitude by the formula A = Ag (1+4kT), 
where Ao and k are constants that the transmitter and receiver must both know. 

If we had two numbers we wanted to send at the same time, we could modulate the sinusoid’s frequency 
as well as its amplitude. This modulation scheme assumes we can estimate the sinusoid’s amplitude and 
frequency; we shall learn that this is indeed possible. 

Now suppose we have a sequence of parameters to send. We have exploited all of the sinusoid’s two 
parameters. What we can do is modulate them for a limited time (say T seconds), and send two parameters 
every T. This simple notion corresponds to how a modem works. Here, typed characters are encoded into 
eight bits, and the individual bits are encoded into a sinusoid’s amplitude and frequency. We'll learn how 
this is done in subsequent modules, and more importantly, we’ll learn what the limits are on such digital 
communication schemes. 


1.5 Introduction Problems’ 


Problem 1.1: RMS Values 
The rms (root-mean-square) value of a periodic signal is defined to be 


| T 
rms[s] = zf s? (t) dt 


where T is defined to be the signal’s period: the smallest positive number such that s(t) = s (t+ 7). 


(a) What is the period of s (t) = Asin (27 fot + $)? 

(b) What is the rms value of this signal? How is it related to the peak value? 

(c) What is the period and rms value of the depicted (Figure 1.5) square wave, generically denoted by 
sq (t)? 

(d) By inspecting any device you plug into a wall socket, you’ll see that it is labeled “110 volts AC.” What 
is the expression for the voltage provided by a wall socket? What is its rms value? 


Problem 1.2: Modems 

The word “modem” is short for “modulator-demodulator.” Modems are used not only for connecting com- 
puters to telephone lines, but also for connecting digital (discrete-valued) sources to generic channels. In this 
problem, we explore a simple kind of modem, in which binary information is represented by the presence or 
absence of a sinusoid (presence representing a “1” and absence a “0”). Consequently, the modem’s transmitted 
signal that represents a single bit has the form 


x(t) = Asin (27 fot) ,O<t<T 
Within each bit interval T, the amplitude is either A or zero. 


°This content is available online at http: //cnx.org/content/m10353/2.17/. 


8 CHAPTER 1. INTRODUCTION 


(a) What is the smallest transmission interval that makes sense with the frequency fo? 

(b) Assuming that ten cycles of the sinusoid comprise a single bit’s transmission interval, what is the 
datarate of this transmission scheme? 

(c) Now suppose instead of using “on-off” signaling, we allow one of several different values for the 
amplitude during any transmission interval. If N amplitude values are used, what is the resulting 
datarate? 

(d) The classic communications block diagram applies to the modem. Discuss how the transmitter must 
interface with the message source since the source is producing letters of the alphabet, not bits. 


Problem 1.3: Advanced Modems 

To transmit symbols, such as letters of the alphabet, RU computer modems use two frequencies (1600 
and 1800 Hz) and several amplitude levels. A transmission is sent for a period of time T (known as the 
transmission or baud interval) and equals the sum of two amplitude-weighted carriers. 


x(t) = A; sin (27 ft) + Agsin (2a fot) ,O<t<T 


We send successive symbols by choosing an appropriate frequency and amplitude combination, and sending 
them one after another. 


(a) What is the smallest transmission interval that makes sense to use with the frequencies given above? 
In other words, what should T be so that an integer number of cycles of the carrier occurs? 

(b) Sketch (using Matlab) the signal that modem produces over several transmission intervals. Make sure 
you axes are labeled. 

(c) Using your signal transmission interval, how many amplitude levels are needed to transmit ASCII 
characters at a datarate of 3,200 bits/s? Assume use of the extended (8-bit) ASCII code. 


NOTE: We use a discrete set of values for A; and Ag. If we have Ny values for amplitude A,, and No 
values for Az, we have N,N» possible symbols that can be sent during each T second interval. To 
convert this number into bits (the fundamental unit of information engineers use to qualify things), 
compute log, (Ni N2). 


Solutions to Exercises in Chapter 1 

Solution to Exercise 1.1 (p. 6) 

As cos (a + 8) = cos (a) cos (8) — sin (a) sin (8), cos (2a (f + 1) n) = cos (27 fn) cos (27) — sin (27 fn) sin (27n) = 
cos (27 fn). 

Solution to Exercise 1.2 (p. 6) 

A square wave takes on the values 1 and —1 alternately. See the plot in Section 2.2.6. 
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Chapter 2 


Signals and Systems 


2.1 Complex Numbers’ 


While the fundamental signal used in electrical engineering is the sinusoid, it can be expressed mathematically 
in terms of an even more fundamental signal: the complex exponential. Representing sinusoids in terms of 
complex exponentials is not a mathematical oddity. Fluency with complex numbers and rational functions 
of complex variables is a critical skill all engineers master. Understanding information and power system 
designs and developing new systems all hinge on using complex numbers. In short, they are critical to 
modern electrical engineering, a realization made over a century ago. 


2.1.1 Definitions 


The notion of the square root of —1 originated with the quadratic formula: the solution of certain quadratic 
equations mathematically exists only if the so-called imaginary quantity /—1 could be defined. Euler? first 
used i for the imaginary unit but that notation did not take hold until roughly Ampére’s time. Ampére® 
used the symbol 7 to denote current (intensité de current). It wasn’t until the twentieth century that the 
importance of complex numbers to circuit theory became evident. By then, using z for current was entrenched 
and electrical engineers chose 7 for writing complex numbers. 

An imaginary number has the form jb = /—b?. A complex number, z, consists of the ordered pair 
(a,b), a is the real component and b is the imaginary component (the j is suppressed because the imaginary 
component of the pair is always in the second position). The imaginary number jb equals (0,b). Note that 
a and 0 are real-valued numbers. 

Figure 2.1 shows that we can locate a complex number in what we call the complex plane. Here, a, 
the real part, is the z-coordinate and 6, the imaginary part, is the y-coordinate. From analytic geometry, we 
know that locations in the plane can be expressed as the sum of vectors, with the vectors corresponding to 
the x and y directions. Consequently, a complex number z can be expressed as the (vector) sum z = a+ jb 
where j indicates the y-coordinate. This representation is known as the Cartesian form of z. An imaginary 
number can’t be numerically added to a real number; rather, this notation for a complex number represents 
vector addition, but it provides a convenient notation when we perform arithmetic manipulations. 

Some obvious terminology. The real part of the complex number z = a+ 7b, written as Re [z], equals 
a. We consider the real part as a function that works by selecting that component of a complex number 
not multiplied by 7. The imaginary part of z, Im[z], equals b: that part of a complex number that is 
multiplied by 7. Again, both the real and imaginary parts of a complex number are real-valued. 

The complex conjugate of z, written as z*, has the same real part as z but an imaginary part of the 


1This content is available online at http://cnx.org/content/m0081/2.27/. 
*http://www-groups.dcs.st-and.ac.uk/~history/Biographies/Euler .html 
Shttp://www-groups.dcs.st-and.ac.uk/~history/Biographies/Ampere . html 
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(a,b) = a+jb =rZa 


a 


Figure 2.1: A complex number is an ordered pair (a,b) that can be regarded as coordinates in the plane. 
Complex numbers can also be expressed in polar coordinates as rZ0. 


opposite sign. 
z = Re[z] + jIm [z] (2.1) 
z* = Re[z] — jIm [z] 


Using Cartesian notation, the following properties easily follow. 


e If we add two complex numbers, the real part of the result equals the sum of the real parts and the 
imaginary part equals the sum of the imaginary parts. This property follows from the laws of vector 
addition. 

ay + jb; + ag + jbe = aj, + ag + 7 (by + bz) 
In this way, the real and imaginary parts remain separate. 

e The product of 7 and a real number is an imaginary number: ja. The product of 7 and an imaginary 
number is a real number: j (jb) = —b because j? = —1. Consequently, multiplying a complex number 
by j rotates the number’s position by 90 degrees. 


Exercise 2.1 (Solution on p. 30.) 
Use the definition of addition to show that the real and imaginary parts can be expressed as a 
sum/difference of a complex number and its conjugate. Re[z] = ate" and Im [z] = a 
Complex numbers can also be expressed in an alternate form, polar form, which we will find quite useful. 
Polar form arises arises from the geometric interpretation of complex numbers. The Cartesian form of a 


complex number can be re-written as 


a b 
+ jb = Va? +O +3 
~ J " (= inzcs) 


By forming a right triangle having sides a and b, we see that the real and imaginary parts correspond to the 
cosine and sine of the triangle’s base angle. We thus obtain the polar form for complex numbers. 


z=at+jb=rZ0 


r=|z|= /a?4+0? a=rcosé 
6 = arctan (b/a) b=rsiné 
The quantity r is known as the magnitude of the complex number z, and is frequently written as |z|. The 


quantity @ is the complex number’s angle. In using the arc-tangent formula to find the angle, we must take 
into account the quadrant in which the complex number lies. 


Exercise 2.2 (Solution on p. 30.) 
Convert 3 — 27 to polar form. 
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2.1.2 Euler’s Formula 


Surprisingly, the polar form of a complex number z can be expressed mathematically as 
z=re?® (2.2) 


To show this result, we use Euler’s relations that express exponentials with imaginary arguments in terms 
of trigonometric functions. 


e?® = cos0 + jsin@ (2.3) 
j0 1 p—50 j0 _ —50 
cos 9 = coal a cg (2.4) 
2 23 
The first of these is easily derived from the Taylor’s series for the exponential. 
21,2 ge 8 
Substituting 70 for x, we find that 
@°.. 6? @? 
JO J 4 ; | 
Se rnig: ope nape 
because j? = —1, 7? = —j, and j* = 1. Grouping separately the real-valued terms and the imaginary-valued 
ones, 
6? o. 08 
jO ae on | 


The real-valued terms correspond to the Taylor’s series for cos (@), the imaginary ones to sin (@), and Euler’s 
first relation results. The remaining relations are easily derived from the first. Because of the relationship 
r= Va? + b?, we see that multiplying the exponential in (2.3) by a real constant corresponds to setting the 
radius of the complex number by the constant. 


2.1.3 Calculating with Complex Numbers 


Adding and subtracting complex numbers expressed in Cartesian form is quite easy: You add (subtract) the 
real parts and imaginary parts separately. 


(21 a 22) = (a4 a a2) +7 (by ea bg) (2.5) 


To multiply two complex numbers in Cartesian form is not quite as easy, but follows directly from following 
the usual rules of arithmetic. 


2122 = (ay + joy) (a2 + jb2) 
= a1a2 — by b2 +3 (ayb2 + azby) 
Note that we are, in a sense, multiplying two vectors to obtain another vector. Complex arithmetic provides 
a unique way of defining vector multiplication. 


(2.6) 


Exercise 2.3 (Solution on p. 30.) 
What is the product of a complex number and its conjugate? 


Division requires mathematical manipulation. We convert the division problem into a multiplication problem 
by multiplying both the numerator and denominator by the conjugate of the denominator. 
zy ay t+ jby 
22 aa + jbo 
ay + jb1 ag — jbo 
ag + jbz az — jbo 
(a1 + jb1) (a2 — jb2) 
7 az? + by” 
G12 + byb2 + j (a2b1 — ayb2) 
7 az? + bo” 
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Because the final result is so complicated, it’s best to remember how to perform division—multiplying 
numerator and denominator by the complex conjugate of the denominator—than trying to remember the 
final result. 

The properties of the exponential make calculating the product and ratio of two complex numbers much 
simpler when the numbers are expressed in polar form. 


229 = 7 p62 = py rye) O14) 


Br | ee ei (01—92) (2.8) 


z= rget92 rg 

To multiply, the radius equals the product of the radii and the angle the sum of the angles. To divide, 
the radius equals the ratio of the radii and the angle the difference of the angles. When the original 
complex numbers are in Cartesian form, it’s usually worth translating into polar form, then performing the 
multiplication or division (especially in the case of the latter). Addition and subtraction of polar forms 
amounts to converting to Cartesian form, performing the arithmetic operation, and converting back to polar 
form. 

Example 2.1 

When we solve circuit problems, the crucial quantity, known as a transfer function, will always be 

expressed as the ratio of polynomials in the variable s = 727 f. What we’ll need to understand the 

circuit’s effect is the transfer function in polar form. For instance, suppose the transfer function 


equals 
s+2 
>— 2. 
s?ts+l1 oy 
s= jaf (2.10) 


Performing the required division is most easily accomplished by first expressing the numerator and 
denominator each in polar form, then calculating the ratio. Thus, 


s+2 | janf+2 
stts+1 —4n2f24 52rf+1 
A + 472 f2 jarctan(n f) 
= ee a ee (2.12) 


va os An? f2)? + 4? f2ei aretan(2m f /(1—4n? f?)) 


(2.11) 


4+ An? f? j(arctan(a f)—arctan(arctan(27f /(1—4? f)))) 
= arctan({ 7 arctan| arctan me Tv 2.1 
Vi — Arf? + 1604 fe « ety) 


2.2 Elemental Signals’ 


Elemental signals are the building blocks with which we build complicated signals. By definition, 
elemental signals have a simple structure. Exactly what we mean by the “structure of a signal” will unfold in 
this section of the course. Signals are nothing more than functions defined with respect to some independent 
variable, which we take to be time for the most part. Very interesting signals are not functions solely of 
time; one great example of which is an image. For it, the independent variables are x and y (two-dimensional 
space). Video signals are functions of three variables: two spatial dimensions and time. Fortunately, most 
of the ideas underlying modern signal theory can be exemplified with one-dimensional signals. 


4This content is available online at http: //cnx.org/content/m0004/2.29/. 
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2.2.1 Sinusoids 


Perhaps the most common real-valued signal is the sinusoid. 
s(t) = Acos (27 fot + ¢) (2.14) 
For this signal, A is its amplitude, fo its frequency, and ¢ its phase. 
2.2.2 Complex Exponentials 
The most important signal is complex-valued, the complex exponential. 


s(t) = Aci 27 fot+¢) 


— Agi? ei2™ fot ty) 


Here, j7 denotes —1. AeJ® is known as the signal’s complex amplitude. Considering the complex 
amplitude as a complex number in polar form, its magnitude is the amplitude A and its angle the signal phase. 
The complex amplitude is also known as a phasor. The complex exponential cannot be further decomposed 
into more elemental signals, and is the most important signal in electrical engineering! Mathematical 
manipulations at first appear to be more difficult because complex-valued numbers are introduced. In fact, 
early in the twentieth century, mathematicians thought engineers would not be sufficiently sophisticated 
to handle complex exponentials even though they greatly simplified solving circuit problems. Steinmetz ° 
introduced complex exponentials to electrical engineering, and demonstrated that “mere” engineers could use 
them to good effect and even obtain right answers! See Section 2.1 for a review of complex numbers and 
complex arithmetic. 

The complex exponential defines the notion of frequency: it is the only signal that contains only one 
frequency component. The sinusoid consists of two frequency components: one at the frequency +/fo and 
the other at — fo. 


EULER RELATION: This decomposition of the sinusoid can be traced to Euler’s relation. 


el2aft fs e Jan ft 


cos (2r ft) = 7 (2.16) 
; el2aft _ e 7 d2rft 

sin (2n ft) = —— (2.17) 

ef?" Ft — cos (Qn ft) + jsin (27 ft) (2.18) 


DECOMPOSITION: The complex exponential signal can thus be written in terms of its real and 
imaginary parts using Euler’s relation. Thus, sinusoidal signals can be expressed as either the real 
or the imaginary part of a complex exponential signal, the choice depending on whether cosine or 
sine phase is needed, or as the sum of two complex exponentials. These two decompositions are 
mathematically equivalent to each other. 


Acos (21 ft + ¢) = Re [Ae?*e??"/"] (2.19) 


Asin (27 ft + ¢) = Im [Ae?*e??"/*] (2.20) 


Using the complex plane, we can envision the complex exponential’s temporal variations as seen in the 
above figure (Figure 2.2). The magnitude of the complex exponential is A, and the initial value of the 
complex exponential at ¢ = 0 has an angle of ¢. As time increases, the locus of points traced by the complex 
exponential is a circle (it has constant magnitude of A). The number of times per second we go around the 
circle equals the frequency f. The time taken for the complex exponential to go around the circle once is 

if 


known as its period T, and equals fF: The projections onto the real and imaginary axes of the rotating 


vector representing the complex exponential signal are the cosine and sine signal of Euler’s relation (2.16). 


Shttp://www.edisontechcenter.org/CharlesProteusSteinmetz. html 
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A sin(2nft+o) 


A cos(2nft+9) 


Figure 2.2: Graphically, the complex exponential scribes a circle in the complex plane as time evolves. 
Its real and imaginary parts are sinusoids. The rate at which the signal goes around the circle is the 
frequency f and the time taken to go around is the period T. A fundamental relationship is T = a 


2.2.3 Real Exponentials 
As opposed to complex exponentials which oscillate, real exponentials (Figure 2.3) decay. 
s(t) =e/7 (2.21) 


The quantity 7 is known as the exponential’s time constant, and corresponds to the time required for 
the exponential to decrease by a factor of 4, which approximately equals 0.368. A decaying complex 
exponential is the product of a real and a complex exponential. 


s(t) = Aei%e—t/ ef? ft 


— Aeive(-l/t+s2nfyt (2.22) 


In the complex plane, this signal corresponds to an exponential spiral. For such signals, we can define 
complex frequency as the quantity multiplying t. 
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Exponential 


Figure 2.3: The real exponential. 


2.2.4 Unit Step 
The unit step function (Figure 2.4) is denoted by u(t), and is defined to be 


0 t<0 
u(t) = {° — (2.23) 


u(t) 


Figure 2.4: The unit step. 


ORIGIN WARNING: This signal is discontinuous at the origin. Its value at the origin need not be 
defined because the value doesn’t matter in signal theory. 


This kind of signal is used to describe signals that “turn on” suddenly. For example, to mathematically repre- 
sent turning on an oscillator, we can write it as the product of a sinusoid and a step: s(t) = Asin (27 ft) u(t). 
2.2.5 Pulse 


The unit pulse (Figure 2.5) describes turning a unit-amplitude signal on for a duration of A seconds, then 
turning it off. 


0, t<0 
po=<1,. bate n (2.24) 
0, t>A 
P,(t) 
1 
A t 


Figure 2.5: The pulse. 


We will find that this is the second most important signal in communications. 
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2.2.6 Square Wave 


The square wave (Figure 2.6) sq(t) is a periodic signal like the sinusoid. It too has an amplitude and a 
period, which must be specified to characterize the signal. We find subsequently that the sine wave is a 
simpler signal than the square wave. 


ASquare Wave 


— Arco — 
——_—_—_—_—_—_—_ | 


Figure 2.6: The square wave. 


2.3 Signal Decomposition*® 


A signal’s complexity is not related to how wiggly it is. Rather, a signal expert looks for ways of decomposing 
a given signal into a sum of simpler signals, which we term the signal decomposition. Though we will 
never compute a signal’s complexity, it essentially equals the number of terms in its decomposition. In 
writing a signal as a sum of component signals, we can change the component signal’s gain by multiplying 
it by a constant and by delaying it. More complicated decompositions could contain derivatives or integrals 
of simple signals. 


Example 2.2 
As an example of signal complexity, we can express the pulse pa (t) as a sum of delayed unit steps. 


pa (t) = u(t) — u(t — A) (2.25) 
Thus, the pulse is a more complex signal than the step. Be that as it may, the pulse is very useful 
to us. 
Exercise 2.4 (Solution on p. 30.) 


Express a square wave having period T and amplitude A as a superposition of delayed and 
amplitude-scaled pulses. 


Because the sinusoid is a superposition of two complex exponentials, the sinusoid is more complex. We could 
not prevent ourselves from the pun in this statement. Clearly, the word “complex” is used in two different 
ways here. The complex exponential can also be written (using Euler’s relation (2.16)) as a sum of a sine and 
a cosine. We will discover that virtually every signal can be decomposed into a sum of complex exponentials, 
and that this decomposition is very useful. Thus, the complex exponential is more fundamental, and Euler’s 
relation does not adequately reveal its complexity. 


2.4 Discrete-Time Signals’ 


So far, we have treated what are known as analog signals and systems. Mathematically, analog signals are 
functions having continuous quantities as their independent variables, such as space and time. Discrete-time 
signals are functions defined on the integers; they are sequences. One of the fundamental results of signal 
theory details the conditions under which an analog signal can be converted into a discrete-time one and 
retrieved without error. This result is important because discrete-time signals can be manipulated by 


6This content is available online at http://cnx.org/content/m0008/2.12/. 
’This content is available online at http: //cnx.org/content/m0009/2.24/. 
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systems instantiated as computer programs. Subsequent modules describe how virtually all analog signal 
processing can be performed with software. 

Discrete-time systems can act on discrete-time signals in ways similar to those found in analog signals 
and systems. Because of the role of software in discrete-time systems, many more different systems can be 
envisioned and “constructed” with programs than can be with analog signals. Consequently, discrete-time 
systems can be easily produced in software, with equivalent analog realizations difficult, if not impossible, 
to design. 

As important as linking analog signals to discrete-time ones may be, discrete-time signals are more 
general, encompassing signals derived from analog ones and signals that aren’t. For example, the characters 
forming a text file form a sequence, which is also a discrete-time signal. We must deal with such symbolic 
valued (p. 153) signals and systems as well. 

As with analog signals, we seek ways of decomposing real-valued discrete-time signals into simpler com- 
ponents. With this approach leading to a better understanding of signal structure, we can exploit that 
structure to represent information (create ways of representing information with signals) and to extract in- 
formation (retrieve the information thus represented). For symbolic-valued signals, the approach is different: 
We develop a common representation of all symbolic-valued signals so that we can embody the information 
they contain in a unified way. From an information representation perspective, the most important issue 
becomes, for both real-valued and symbolic-valued signals, efficiency; What is the most parsimonious and 
compact way to represent information so that it can be extracted later. 


2.4.1 Real- and Complex-valued Signals 


A discrete-time signal is represented symbolically as s(n), where n = {...,—1,0,1,...}. We usually draw 
discrete-time signals as stem plots to emphasize the fact they are functions defined only on the integers. 
We can delay a discrete-time signal by an integer just as with analog ones. A delayed unit sample has the 
expression 6 (n — m), and equals one when n = m. 
Sn 
1 


alae 


Figure 2.7: The discrete-time cosine signal is plotted as a stem plot. Can you find the formula for this 
signal? 


2.4.2 Complex Exponentials 


The most important signal is, of course, the complex exponential sequence. 


s(n) = e)?7fn (2.26) 


2.4.3 Sinusoids 


Discrete-time sinusoids have the obvious form s(n) = Acos(27fn+¢). As opposed to analog complex 


exponentials and sinusoids that can have their frequencies be any real value, frequencies of their discrete- 
time counterparts yield unique waveforms only when f lies in the interval (—5, 4]. This property can be 


easily understood by noting that adding an integer to the frequency of the discrete-time complex exponential 
has no effect on the signal’s value. 
el2m(f+m)n = clan fn ej2nmn 


(2.27) 


= eden fn 
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This derivation follows because the complex exponential evaluated at an integer multiple of 27 equals one. 


2.4.4 Unit Sample 


The second-most important discrete-time signal is the unit sample, which is defined to be 


= {1 aes (2.28) 


0 otherwise 


5n 


O98 1 


Figure 2.8: The unit sample. 


Examination of a discrete-time signal’s plot, like that of the cosine signal shown in Figure 2.7, reveals that 
all discrete-time signals consist of a sequence of delayed and scaled unit samples. Because the value of 
a sequence at each integer m is denoted by s(m) and the unit sample delayed to occur at m is written 
6 (nm —m), we can decompose any signal as a sum of unit samples delayed to the appropriate location and 
scaled by the signal value. 


s(n)= S > s(m)d(n—m) (2.29) 


m>=—Cco 


This kind of decomposition is unique to discrete-time signals, and will prove useful subsequently. 


2.4.5 Symbolic-valued Signals 


Another interesting aspect of discrete-time signals is that their values do not need to be real numbers. We 
do have real-valued discrete-time signals like the sinusoid, but we also have signals that denote the sequence 
of characters typed on the keyboard. Such characters certainly aren’t real numbers, and as a collection of 
possible signal values, they have little mathematical structure other than that they are members of a set. 
More formally, each element of the symbolic-valued signal s (nm) takes on one of the values {a1,...,a« } which 
comprise the alphabet A. This technical terminology does not mean we restrict symbols to being mem- 
bers of the English or Greek alphabet. They could represent keyboard characters, bytes (8-bit quantities), 
integers that convey daily temperature. Whether controlled by software or not, discrete-time systems are 
ultimately constructed from digital circuits, which consist entirely of analog circuit elements. Furthermore, 
the transmission and reception of discrete-time signals, like e-mail, is accomplished with analog signals and 
systems. Understanding how discrete-time and analog signals and systems intertwine is perhaps the main 
goal of this course. 


2.5 Introduction to Systems* 


Signals are manipulated by systems. Mathematically, we represent what a system does by the notation 
y (t) = S [x (t)], with x representing the input signal and y the output signal. 


8This content is available online at http: //cnx.org/content/m0005/2.19/. 
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——> System ~—> 


Figure 2.9: The system depicted has input x(t) and output y (¢). Mathematically, systems operate on 
function(s) to produce other function(s). In many ways, systems are like functions, rules that yield a 
value for the dependent variable (our output signal) for each value of its independent variable (its input 
signal). The notation y (¢) = S [x (t)] corresponds to this block diagram. We term S [-] the input-output 
relation for the system. 


This notation mimics the mathematical symbology of a function: A system’s input is analogous to an 
independent variable and its output the dependent variable. For the mathematically inclined, a system is a 
functional: a function of a function (signals are functions). 

Simple systems can be connected together—one system’s output becomes another’s input-to accomplish 
some overall design. Interconnection topologies can be quite complicated, but usually consist of weaves of 
three basic interconnection forms. 


2.5.1 Cascade Interconnection 


x(t) w(t) y(t) 


Silt] -——>| Salt] ——> 


Figure 2.10: Interconnecting systems so that one system’s output serves as the input to another is the 
cascade configuration. 


The simplest form is when one system’s output is connected only to another’s input. Mathematically, 
w (t) = Sj [a (t)], and y(t) = S> [w (£)], with the information contained in x (t) processed by the first, then 
the second system. In some cases, the ordering of the systems matter, in others it does not. For example, in 
the fundamental model of communication (Figure 1.3) the ordering most certainly matters. 


2.5.2 Parallel Interconnection 


mee S41 [*] 


—_> SI] Ly 


x(t) 


Figure 2.11: The parallel configuration. 


A signal x(t) is routed to two (or more) systems, with this signal appearing as the input to all systems 
simultaneously and with equal strength. Block diagrams have the convention that signals going to more 
than one system are not split into pieces along the way. Two or more systems operate on x(t) and their 
outputs are added together to create the output y (t). Thus, y (¢) = Sj |x (¢)|+ S2 [x (t)], and the information 
in x(t) is processed separately by both systems. 
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2.5.3 Feedback Interconnection 


pe si} 2% 


Sal] 


Figure 2.12: The feedback configuration. 


The subtlest interconnection configuration has a system’s output also contributing to its input. Engineers 
would say the output is “fed back” to the input through system 2, hence the terminology. The mathematical 
statement of the feedback interconnection (Figure 2.12) is that the feed-forward system produces the output: 
y (t) = Sy [e(t)]. The input e (¢) equals the input signal minus the output of some other system’s output to 
y(t): e(t) = a(t) — $9 [y (t)]. Feedback systems are omnipresent in control problems, with the error signal 
used to adjust the output to achieve some condition defined by the input (controlling) signal. For example, 
in a car’s cruise control system, x (t) is a constant representing what speed you want, and y (t) is the car’s 
speed as measured by a speedometer. In this application, system 2 is the identity system (output equals 
input). 


2.6 Simple Systems’ 


Systems manipulate signals, creating output signals derived from their inputs. Why the following are cate- 
gorized as “simple” will only become evident towards the end of the course. 


2.6.1 Sources 


Sources produce signals without having input. We like to think of these as having controllable parameters, 
like amplitude and frequency. Examples would be oscillators that produce periodic signals like sinusoids and 
square waves and noise generators that yield signals with erratic waveforms (more about noise subsequently). 
Simply writing an expression for the signals they produce specifies sources. A sine wave generator might 
be specified by y(t) = Asin (27 fot) u(t), which says that the source was turned on at t = 0 to produce a 
sinusoid of amplitude A and frequency fo. 


2.6.2 Amplifiers 


An amplifier (Figure 2.13) multiplies its input by a constant known as the amplifier gain. 


y (t) = Ga (t) (2.30) 


ee 


; sats 


Figure 2.13: An amplifier. 


°This content is available online at http: //cnx.org/content/m0006/2.24/. 
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The gain can be positive or negative (if negative, we would say that the amplifier inverts its input) and 
can be greater than one or less than one. If less than one, the amplifier actually attenuates. A real-world 
example of an amplifier is your home stereo. You control the gain by turning the volume control. 


2.6.3 Delay 


A system serves as a time delay (Figure 2.14) when the output signal equals the input signal at an earlier 
time. 


y(t) =x (t—7) (2.31) 
Delay T 
——> t /——_> 


Figure 2.14: A delay. 


Here, 7 is the delay. The way to understand this system is to focus on the time origin: The output at time 
t = 7 equals the input at time t = 0. Thus, if the delay is positive, the output emerges later than the input, 
and plotting the output amounts to shifting the input plot to the right. The delay can be negative, in which 
case we say the system advances its input. Such systems are difficult to build (they would have to produce 
signal values derived from what the input will be), but we will have occasion to advance signals in time. 


2.6.4 Time Reversal 


With a time-reversal system, the output signal equals the input signal flipped about the vertical axis (the 
time origin). 


y (t) = «(-t) (2.32) 


eo 


Time 


—— >| Reverse [> 


Figure 2.15: A time reversal system. 


Again, such systems are difficult to build, but the notion of time reversal occurs frequently in communications 
systems. 
Exercise 2.5 (Solution on p. 30.) 
Mentioned earlier was the issue of whether the ordering of systems mattered. In other words, if 
we have two systems in cascade, does the output depend on which comes first? Determine if the 
ordering matters for the cascade of an amplifier and a delay and for the cascade of a time-reversal 
system and a delay. 


2.6.5 Derivative Systems and Integrators 


Systems that perform calculus-like operations on their inputs can produce waveforms significantly different 
than present in the input. Derivative systems operate in a straightforward way: A first-derivative system 
would have the input-output relationship y (t) = tr (t). Integral systems have the complication that the 
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integral’s limits must be defined. It is a signal theory convention that the elementary integral operation have 
a lower limit of —oo, and that the value of all signals at t = —oo equals zero. A simple integrator would 
have input-output relation 


y(t) = / x(a) da (2.33) 


2.6.6 Linear Systems 


Linear systems are a class of systems rather than having a specific input-output relation. Linear systems 
form the foundation of system theory, and are the most important class of systems in communications. They 
have the property that when the input is expressed as a weighted sum of component signals, the output 
equals the same weighted sum of the outputs produced by each component. When S [-] is linear, 


S [Gya1 (t) + Gove (t)| = GiS [x1 (t)] + GoS [x2 (t)] (2.34) 


for all choices of signals and gains. 
This general input-output relation property can be manipulated to indicate specific properties shared by 
all linear systems. 


e S(Gzx (t)] = GS [x (t)] The colloquialism summarizing this property is “Double the input, you double 
the output.” Note that this property is consistent with alternate ways of expressing gain changes: 
Since 22 (t) also equals x (t) + #(t), the linear system definition provides the same output no matter 
which of these is used to express a given signal. 

e S(0] = 0 If the input is identically zero for all time, the output of a linear system must be zero. 
This property follows from the simple derivation S [0] = S [x (t) — x (t)] = S [x (t)] — S [x (t)] = 0. 


Just why linear systems are so important is related not only to their properties, which are divulged throughout 
this course, but also because they lend themselves to relatively simple mathematical analysis. Said another 
way, “They’re the only systems we thoroughly understand!” 

We can find the output of any linear system to a complicated input by decomposing the input into simple 
signals. The equation above (2.34) says that when a system is linear, its output to a decomposed input is 
the sum of outputs to each input. For example, if 


x(t) =e -* +sin (27 fot) 
the output S (x (t)) of any linear system equals 


y(t) = S [e~'] + S [sin (27 fot)] 


2.6.7 Time-Invariant Systems 


Systems that don’t change their input-output relation with time are said to be time-invariant. The mathe- 
matical way of stating this property is to use the signal delay concept described in Section 2.6.3. 


y(t) = S[e(Q] > y(t—7) =S[e(t—7) (2.35) 


If you delay (or advance) the input, the output is similarly delayed (advanced). Thus, a time-invariant 
system responds to an input you may supply tomorrow the same way it responds to the same input applied 
today; today’s output is merely delayed to occur tomorrow. 

The collection of linear, time-invariant systems are the most thoroughly understood systems. Much of 
the signal processing and system theory discussed here concentrates on such systems. For example, electric 
circuits are, for the most part, linear and time-invariant. Nonlinear ones abound, but characterizing them 
so that you can predict their behavior for any input remains an unsolved problem. 
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Input-Output Relation | Linear | Time-Invariant 
y(t) = £(a) yes yes 
y(t) = 4 (a) yes yes 
y(t) = (4(a))” no yes 
y(t) = 4(a) +2 yes yes 
y(t) = 21 +29 yes yes 
y(t) =a(t—T) yes yes 
y (t) = cos (27 ft) x (t) yes no 
y (t) = a (-t) yes no 
y (t) = x? (t) no yes 
y(t) = |e (t)| no yes 
y (t) = ma (t) +6 no yes 
Table 2.1 


2.7 Signals and Systems Problems" 


Problem 2.1: Complex Number Arithmetic 
Find the real part, imaginary part, the magnitude and angle of the complex numbers given by the following 
expressions. 


(a) -1 
1+ V3j 
(b) 24 
(c) 1+jt+e2 
(d) 3 +e3™ +6495 


Problem 2.2: Discovering Roots 
Complex numbers expose all the roots of real (and complex) numbers. For example, there should be two 
square-roots, three cube-roots, etc. of any number. Find the following roots. 


(a) What are the cube-roots of 27? In other words, what is 273? 
(b) What are the fifth roots of 3 (35)? 
(c) What are the fourth roots of one? 


Problem 2.3: Cool Exponentials 
Simplify the following (cool) expressions. 


(a) j7 

(b) j¥ 

(e)ig? 
Problem 2.4: Complex-valued Signals 


Complex numbers and phasors play a very important role in electrical engineering. Solving systems for 
complex exponentials is much easier than for sinusoids, and linear systems analysis is particularly easy. 


10This content is available online at http: //cnx.org/content/m10348/2.27/. 
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(a) Express each as a sum of complex exponentials. Also, re-express each as the real and imaginary parts 
of a complex exponential. What is the frequency (in Hz) of each? In general, are your answers unique? 
If so, prove it; if not, find an alternative answer for the complex exponential representation. 
i) 3sin (24t) 
ii) V2cos (2760t + 7) 
iii) 2cos (¢+ §) + 4sin (t — 3) 
(b) Show that for linear systems having real-valued outputs for real inputs, that when the input is the 
real part of a complex exponential, the output is the real part of the system’s output to the complex 
exponential (see Figure 2.16). 


S [Re { Aci?"Ft\] = Re {S | Ae??*F*]} 


Aelentt S[Re[Ael27it]] 
—»| Ret] -»} gy > 


Ael2ntt Re[S[Ael2nit]] 
—> FS) > Rel) —r 


Figure 2.16 


Problem 2.5: 

Express each of the indicated voltages as the real part of a complex exponential: v (t) = Re [Ve*]. Explicitly 
indicate the value of the complex amplitude V and the complex frequency s. Represent each complex 
amplitude as a vector in the V-plane, and indicate the location of the frequencies in the complex s-plane. 


(a) v (t) = cos (5t) (b) v(t) = sin (8¢ + 4) 
(c) v(t) =e"* (d) v(t) = e7*' sin (4t + 32) 
) = 5e”* sin (8t + 277) (f) v(t) =-2 
) =4sin (2t) + 3cos(2t) (h) v(t) = 2cos (1007t + £) — V3sin (1007¢ + 3) 
Problem 2.6: 


Express each of the depicted signals (Figure 2.17) as a linear combination of delayed and weighted step 
functions and ramps (the integral of a step). 


27 


10; Ag 


n= = 


Figure 2.17 


Problem 2.7: Linear, Time-Invariant Systems 
When the input to a linear, time-invariant system is the signal x (t), the output is the signal y (t) (Figure 2.18). 


(a) Find and sketch this system’s output when the input is the depicted signal (Figure 2.19). 
(b) Find and sketch this system’s output when the input is a unit step. 
Ax(t) Ayit) 
1 —_______ 


Figure 2.18 
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Figure 2.21 


Problem 2.8: Linear Systems 
The depicted input (Figure 2.20a) x(t) to a linear, time-invariant system yields the output y (t). 


(a) What is the system’s output to a unit step input u(t)? 
(b) What will the output be when the input is the depicted square wave (Figure 2.20b)? 


Problem 2.9: Communication Channel 
A particularly interesting communication channel can be modeled as a linear, time-invariant system. When 
the transmitted signal x(t) is a pulse, the received signal r (t) is as shown in Figure 2.21. 


(a) What will be the received signal when the transmitter sends the pulse sequence xj (t) shown at the 
top of Figure 2.22? 

(b) What will be the received signal when the transmitter sends the pulse signal x2 (t) shown at the bottom 
of Figure 2.22 that has half the duration as the original? 
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x4 (t) 
{ 
eee 
ei 
1 2 3 
xa(t) 
{ 
t 
1/2 1 
Figure 2.22 


Problem 2.10: Analog Computers 
So-called analog computers use circuits to solve mathematical problems, particularly when they involve 
differential equations. Suppose we are given the following differential equation to solve. 


Su lt) tay (t) = a(t) 


In this equation, a is a constant. 


(a) When the input is a unit step (x (t) = u(t)), the output is given by y(t) = (1 — e~*’) u(t). What is the 
total energy expended by the input? 

(b) Instead of a unit step, suppose the input is a unit pulse (unit-amplitude, unit-duration) delivered to 
the circuit at time t = 10. What is the output voltage in this case? Sketch the waveform. 
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Solutions to Exercises in Chapter 2 


Solution to Exercise 2.1 (p. 12) 

z+2* =a+jb+a-— jb = 2a = 2Re[z]. Similarly, z — 2* = a+ jb— (a — jb) = 27b = 271m [z] 

Solution to Exercise 2.2 (p. 12) 

To convert 3 — 27 to polar form, we first locate the number in the complex plane in the fourth quadrant. 


The distance from the origin to the complex number is the magnitude r, which equals V13 = 4/3? + 2). 


The angle equals — arctan (3) or —0.588 radians (—33.7 degrees). The final answer is V13Z (—33.7) degrees. 
Solution to Exercise 2.3 (p. 13) 
z2* = (a+ jb) (a— jb) = a2 +B. Thus, z2* =r? = |z|’. 
Solution to Exercise 2.4 (p. 18) 
sq (t) = na—co (—1)"Apr/2 (t— 2) 
Solution to Exercise 2.5 (p. 23) 
In the first case, order does not matter; in the second it does. “Delay” means t — t — 7. “Time-reverse” 
means t + —t 
Case 1 y(t) = Gx (t — 7), and the way we apply the gain and delay the signal gives the same result. 
Case 2 Time-reverse then delay: y(t) = «(—(t—7)) = #(—t+7). Delay then time-reverse: y(t) = 
x(-t—T). 


Chapter 3 


Analog Signal Processing 


3.1 Voltage, Current, and Generic Circuit Elements’ 


We know that information can be represented by signals; now we need to understand how signals are 
physically realized. Over the years, electric signals have been found to be the easiest to use. Voltage and 
currents comprise the electric instantiations of signals. Thus, we need to delve into the world of electricity 
and electromagnetism. The systems used to manipulate electric signals directly are called circuits, and they 
refine the information representation or extract information from the voltage or current. In many cases, they 
make nice examples of linear systems. 

A generic circuit element places a constraint between the classic variables of a circuit: voltage and current. 
Voltage is electric potential and represents the “push” that drives electric charge from one place to another. 
What causes charge to move is a physical separation between positive and negative charge. A battery 
generates, through electrochemical means, excess positive charge at one terminal and negative charge at the 
other, creating an electric field. Voltage is defined across a circuit element, with the positive sign denoting 
a positive voltage drop across the element. When a conductor connects the positive and negative potentials, 
current flows, with positive current indicating that positive charge flows from the positive terminal to the 
negative. Electrons comprise current flow in many cases. Because electrons have a negative charge, electrons 
move in the opposite direction of positive current flow: Negative charge flowing to the right is equivalent to 
positive charge moving to the left. 

It is important to understand the physics of current flow in conductors to appreciate the innovation of 
new electronic devices. Electric charge can arise from many sources, the simplest being the electron. When 
we say that “electrons flow through a conductor,” what we mean is that the conductor’s constituent atoms 
freely give up electrons from their outer shells. “Flow” thus means that electrons hop from atom to atom 
driven along by the applied electric potential. A missing electron, however, is a virtual positive charge. 
Electrical engineers call these holes, and in some materials, particularly certain semiconductors, current 
flow is actually due to holes. Current flow also occurs in nerve cells found in your brain. Here, neurons 
“communicate” using propagating voltage pulses that rely on the flow of positive ions (potassium and sodium 
primarily, and to some degree calcium) across the neuron’s outer wall. Thus, current can come from many 
sources, and circuit theory can be used to understand how current flows in reaction to electric fields. 


1This content is available online at http://cnx.org/content/m0011/2.14/. 
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iv, 


Figure 3.1: The generic circuit element. 


Current flows through circuit elements, such as that depicted in Figure 3.1, and through conductors, 
which we indicate by lines in circuit diagrams. For every circuit element we define a voltage and a current. 
The element has a v-i relation defined by the element’s physical properties. In defining the v-i relation, we 
have the convention that positive current flows from positive to negative voltage drop. Voltage has units of 
volts, and both the unit and the quantity are named for Alessandro Volta”. Current has units of amperes, 
and is named for the French physicist André-Marie Ampére?®. 

Voltages and currents also carry power. Again using the convention shown in Figure 3.1 for circuit 
elements, the instantaneous power at each moment of time consumed by the element is given by the 
product of the voltage and current. 

p(t) =v (tilt) 


A positive value for power indicates that at time t the circuit element is consuming power; a negative value 
means it is producing power. With voltage expressed in volts and current in amperes, power defined this 
way has units of watts. Just as in all areas of physics and chemistry, power is the rate at which energy is 
consumed or produced. Consequently, energy is the integral of power. 


Again, positive energy corresponds to consumed energy and negative energy corresponds to energy produc- 
tion. Note that a circuit element having a power profile that is both positive and negative over some time 
interval could consume or produce energy according to the sign of the integral of power. The units of energy 
are joules since a watt equals joules/second. 

Exercise 3.1 (Solution on p. 94.) 

Residential energy bills typically state a home’s energy usage in kilowatt-hours. Is this really a unit 

of energy? If so, how many joules equals one kilowatt-hour? 


3.2 Ideal Circuit Elements‘ 


The elementary circuit elements—the resistor, capacitor, and inductor— impose linear relationships between 
voltage and current. 


3.2.1 Resistor 


Figure 3.2: Resistor. v = Ri 


*https: //nationalmaglab. org/education/magnet -academy/history-of-electricity-magnetism/pioneers/ 
alessandro-volta 

Shttp://www-groups.dcs.st-and.ac.uk/~history/Biographies/Ampere .html 

“This content is available online at http://cnx.org/content/m0012/2.21/. 
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The resistor is far and away the simplest circuit element. In a resistor, the voltage is proportional to the 
current, with the constant of proportionality R, known as the resistance. 


v(t) = Ri(t) 


Resistance has units of ohms, denoted by 2, named for the German electrical scientist Georg Ohm°. Some- 
times, the v-i relation for the resistor is written 1 = Gv, with G, the conductance, equal to Fe Conductance 
has units of Siemens (S), and is named for the German electronics industrialist Werner von Siemens°. 

When resistance is positive, as it is in most cases, a resistor consumes power. A resistor’s instantaneous 
power consumption can be written one of two ways. 

p(t) = Ri? (t) = pv" (t) 

As the resistance approaches infinity, we have what is known as an open circuit: No current flows but a 
non-zero voltage can appear across the open circuit. As the resistance becomes zero, the voltage goes to zero 
for a non-zero current flow. This situation corresponds to a short circuit. A superconductor physically 
realizes a short circuit. 


3.2.2 Capacitor 


d 
Figure 3.3: Capacitor. 1 = Ca (t) 


The capacitor stores charge and the relationship between the charge stored and the resultant voltage is 
q = Cv. The constant of proportionality, the capacitance, has units of farads (F), and is named for the 
English experimental physicist Michael Faraday’. As current is the rate of change of charge, the v-i relation 
can be expressed in differential or integral form. 


i@=CFv) o v=7 f ila)da (3.1) 


If the voltage across a capacitor is constant, then the current flowing into it equals zero. In this situation, 
the capacitor is equivalent to an open circuit. The power consumed/produced by a voltage applied to a 
capacitor depends on the product of the voltage and its derivative. 


p(t) = Cv(t) Soi) 


This result means that a capacitor’s total energy expenditure up to time t¢ is concisely given by 


This expression presumes the fundamental assumption of circuit theory: all voltages and currents in 
any circuit were zero in the far distant past (t = —oo). 


Shttp://www-groups.dcs.st-and.ac.uk/~history/Biographies/Ohm. html 
Shttps: //www.siemens.com/history/en/personalities/founder_generation.htm 
Thttps://en.wikipedia. org/wiki/Michael_Faraday 
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3.2.3 Inductor 


d 
Figure 3.4: Inductor. v = Li (t) 


The inductor stores magnetic flux, with larger valued inductors capable of storing more flux. Inductance has 
units of henries (H), and is named for the American physicist Joseph Henry®. The differential and integral 
forms of the inductor’s v-i relation are 


v(t) = Li (t) or i(t) = z/ uv (a) da (3.2) 


The power consumed/produced by an inductor depends on the product of the inductor current and its 
derivative 


p(t) = Li(t) <i(t) 


and its total energy expenditure up to time ¢ is given by 


3.2.4 Sources 


Figure 3.5: The voltage source on the left and current source on the right are like all circuit elements 
in that they have a particular relationship between the voltage and current defined for them. For the 
voltage source, v = vs for any current 2; for the current source, 7 = —i; for any voltage v. 


Sources of voltage and current are also circuit elements, but they are not linear in the strict sense of linear 
systems. For example, the voltage source’s v-i relation is v = vs regardless of what the current might be. 
As for the current source, i = —i, regardless of the voltage. Another name for a constant-valued voltage 
source is a battery, and can be purchased in any supermarket. Current sources, on the other hand, are much 
harder to acquire; we’ll learn why later. 


3.3 Ideal and Real-World Circuit Elements’ 


Source and linear circuit elements are ideal circuit elements. One central notion of circuit theory is combining 
the ideal elements to describe how physical elements operate in the real world. For example, the 1 kQ 


Shttp://siarchives.si.edu/history/exhibits/joseph-henry 
°This content is available online at http: //cnx.org/content/m0013/2.9/. 
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ly_+ Vi - 
R; lout 
+ + 
+ 
Vin C) Ro Vour. “in CC) Vout 
(a) (b) 
Vin(t) Vout(t) 


Source }+———>_ System }+———_> 


(c) 


Figure 3.6: The circuit shown in the top two figures is perhaps the simplest circuit that performs a 
signal processing function. On the bottom is the block diagram that corresponds to the circuit. The 
input is provided by the voltage source vj, and the output is the voltage vout across the resistor label Ro. 
As shown in the middle, we analyze the circuit—understand what it accomplishes—by defining currents 
and voltages for all circuit elements, and then solving the circuit and element equations. 


resistor you can hold in your hand is not exactly an ideal 1 kQ resistor. First of all, physical devices 
are manufactured to close tolerances (the tighter the tolerance, the more money you pay), but never have 
exactly their advertised values. The fourth band on resistors specifies their tolerance; 10% is common. More 
pertinent to the current discussion is another deviation from the ideal: If a sinusoidal voltage is placed across 
a physical resistor, the current will not be exactly proportional to it as frequency becomes high, say above 
1 MHz. At very high frequencies, the way the resistor is constructed introduces inductance and capacitance 
effects. Thus, the smart engineer must be aware of the frequency ranges over which his ideal models match 
reality well. 

On the other hand, physical circuit elements can be readily found that well approximate the ideal, but 
they will always deviate from the ideal in some way. For example, a flashlight battery, like a C-cell, roughly 
corresponds to a 1.5 V voltage source. However, it ceases to be modeled by a voltage source capable of 
supplying any current (that’s what ideal ones can do!) when the resistance of the light bulb is too small. 


3.4 Electric Circuits and Interconnection Laws” 


A circuit connects circuit elements together in a specific configuration designed to transform the source 
signal (originating from a voltage or current source) into another signal—the output—that corresponds to 
the current or voltage defined for a particular circuit element. A simple resistive circuit is shown in Figure 3.6. 
This circuit is the electrical embodiment of a system having its input provided by a source system producing 
Vin (t). 

To understand what this circuit accomplishes, we want to determine the voltage across the resistor labeled 
by its value Rg. Recasting this problem mathematically, we need to solve some set of equations so that we 
relate the output voltage vout to the source voltage. It would be simple—a little too simple at this point—if 
we could instantly write down the one equation that relates these two voltages. Until we have more knowledge 
about how circuits work, we must write a set of equations that allow us to find all the voltages and currents 
that can be defined for every circuit element. Because we have a three-element circuit, we have a total of six 
voltages and currents that must be either specified or determined. You can define the directions for current 
flow and positive voltage drop any way you like. When two people solve a circuit their own ways, the 
signs of their variables may not agree, but current flow and voltage drop values for each element will agree. 
Do recall in defining your voltage and current variables that the v-i relations for the elements presume that 


10This content is available online at http: //cnx.org/content/m0014/2.27/. 
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positive current flow is in the same direction as positive voltage drop. Once you define voltages and currents, 
we need six non-redundant equations to solve for the six unknown voltages and currents. By specifying the 
source, we have one; this amounts to providing the source’s v-i relation. The v-i relations for the resistors 
give us two more. We are only halfway there; where do we get the other three equations we need? 

What we need to solve every circuit problem are mathematical statements that express how the circuit 
elements are interconnected. Said another way, we need the laws that govern the electrical connection of 
circuit elements. First of all, the places where circuit elements attach to each other are called nodes. 
Two nodes are explicitly indicated in Figure 3.6; a third is at the bottom where the voltage source and 
resistor Ry are connected. Electrical engineers tend to draw circuit diagrams—schematics— in a rectilinear 
fashion. Thus the long line connecting the bottom of the voltage source with the bottom of the resistor 
is intended to make the diagram look pretty. This line simply means that the two elements are connected 
together. Kirchhoff’s Laws, one for voltage and one for current, determine what a connection among 
circuit elements means. These laws can help us analyze this circuit. 


3.4.1 Kirchhoff’s Current Law 


At every node, the sum of all currents entering a node must equal zero. What this law means physically is 
that charge cannot accumulate in a node; what goes in must come out. In our example circuit depicted in 
Figure 3.6, we have a three-node circuit and thus have three KCL equations. 


11 — ig =0 
itig=0 


Note that the current entering a node is the negative of the current leaving the node. 

Given any two of these KCL equations, we can find the other by adding or subtracting them. Thus, one 
of them is redundant and, in mathematical terms, we can discard any one of them. The convention is to 
discard the equation for the (unlabeled) node at the bottom of the circuit. 

Exercise 3.2 (Solution on p. 94.) 
In writing KCL equations, you will find that in an n-node circuit, exactly one of them is always 
redundant. Can you sketch a proof of why this might be true? 

Hint: It has to do with the fact that charge won’t accumulate in one place on its own. 


3.4.2 Kirchhoff’s Voltage Law (KVL) 


The voltage law says that the sum of voltages around every closed loop in the circuit must equal zero. A 
closed loop has the obvious definition: Starting at a node, trace a path through the circuit that returns you 
to the origin node. KVL expresses the fact that electric fields are conservative: The total work performed 
in moving a test charge around a closed path is zero. The KVL equation for our circuit is 


vy tvg—-—v=0 


In writing KVL equations, we follow the convention that an element’s voltage enters with a plus sign when 
traversing the closed path, we go from the positive to the negative of the voltage’s definition. 

For the example circuit shown in Figure 3.6 (page 35), we have three v-i relations, two KCL equations, 
and one KVL equation for solving for the circuit’s six voltages and currents. 


v-l: Vv = Vin KCL: —i—7, =0 KVL: —v + v1 + Vout = 0 
yy = Rity 41 — tout = 0 
Vout = Ratout 


We have exactly the right number of equations! Eventually, we will discover shortcuts for solving circuit 
problems; for now, we want to eliminate all the variables but voy; and determine how it depends on v;, and 
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on resistor values. The KVL equation can be rewritten as vin = V1 + Vout. Substituting into it the resistor’s 
v-i relation, we have vin = Rit, + Rotour. Yes, we temporarily eliminate the quantity we seek. Though 
not obvious, it is the simplest way to solve the equations. One of the KCL equations says 41 = tout, which 
means that vin = Ritout + Rotout = (Ri + Re) tout. Solving for the current in the output resistor, we have 
7 Vin 
fovt = Ft Be 
sources and circuit-element values. To find any other circuit quantities, we can back substitute this answer 
into our original equations or ones we developed along the way. Using the v-i relation for the output resistor, 
we obtain the quantity we seek. 


. We have now solved the circuit: We have expressed one voltage or current in terms of 


R 
Vout = z Vin 
Ri + Ro 


Exercise 3.3 (Solution on p. 94.) 
Referring back to Figure 3.6 (page 35), a circuit should serve some useful purpose. What kind of 
system does our circuit realize and, in terms of element values, what are the system’s parameter(s)? 


3.5 Power Dissipation in Resistor Circuits” 


We can find voltages and currents in simple circuits containing resistors and voltage or current sources. We 
should examine whether these circuits variables obey the Conservation of Power principle: since a circuit is 
a closed system, it should not dissipate or create energy. For the moment, our approach is to investigate 
first a resistor circuit’s power consumption/creation. Later, we will prove that because of KVL and KCL 
all circuits conserve power. 

As defined on p. 32, the instantaneous power consumed/created by every circuit element equals the 
product of its voltage and current. The total power consumed/created by a circuit equals the sum of each 


element’s power. 
P= ) Uktk 
k 


Recall that each element’s current and voltage must obey the convention that positive current is defined to 
enter the positive-voltage terminal. With this convention, a positive value of vgiz corresponds to consumed 
power, a negative value to created power. Because the total power in a circuit must be zero (P = 0), some 
circuit elements must create power while others consume it. 

Consider the simple series circuit appearing in Figure 3.6 (page 35). In performing our calculations, 
we defined the current joy; to flow through the positive-voltage terminals of both resistors and found it to 


‘Ds 
equal ious = Rik The voltage across the resistor R2 is the output voltage and we found it to equal 
1 2 
R 
Vout = io Consequently, calculating the power for this resistor yields 
1 2 


as 
(Ri Boy in 


Py = 


Consequently, this resistor dissipates power because P, is positive. This result should not be surprising since 
we showed (p. 33) that the power consumed by any resistor equals either of the following. 
2 

vu 2 
— or «RR 3.3 

, (3.3) 
Since resistors are positive-valued, resistors always dissipate power. But where does a resistor’s power 
go? By Conversation of Power, the dissipated power must be absorbed somewhere. The answer is not 
directly predicted by circuit theory, but is by physics. Current flowing through a resistor makes it hot; its 
power is dissipated by heat. 


11This content is available online at http: //cnx.org/content/m17305/1.5/. 


38 CHAPTER 3. ANALOG SIGNAL PROCESSING 


iy + Vi - 
R, i iout 
+ + 
+ 
(a) (b) 


Figure 3.7: The circuit shown is perhaps the simplest circuit that performs a signal processing function. 
The input is provided by the voltage source vin and the output is the voltage vout across the resistor 
labelled Ro. 


NOTE: A physical wire has a resistance and hence dissipates power (it gets warm just like a resistor 
in a circuit). In fact, the resistance of a wire of length L and cross-sectional area A is given by 


pL 
A 
The quantity p is known as the resistivity and presents the resistance of a unit-length unit cross- 
sectional area material constituting the wire. Resistivity has units of ohm-meters. Most materials 
have a positive value for p, which means the longer the wire, the greater the resistance and thus 
the power dissipated. The thicker the wire, the smaller the resistance. Superconductors have zero 
resistivity and hence do not dissipate power. If a room-temperature superconductor could be found, 
electric power could be sent through power lines without loss! 


R 


Exercise 3.4 (Solution on p. 94.) 
Calculate the power consumed/created by the resistor R, in our simple circuit example. 


We conclude that both resistors in our example circuit consume power, which points to the voltage source 
as the producer of power. The current flowing into the source’s positive terminal is —ioyt. Consequently, 
the power calculation for the source yields 
1 2 
eee ee? 
Ri + Re oe 


We conclude that the source provides the power consumed by the resistors, no more, no less. 


—Vintout = 


Exercise 3.5 (Solution on p. 94.) 
Confirm that the source produces exactly the total power consumed by both resistors. 


This result is quite general: sources produce power and the circuit elements, especially resistors, consume it. 
But where do sources get their power? Again, circuit theory does not model how sources are constructed, 
but the theory decrees that all sources must be provided energy to work. 


3.6 Series and Parallel Circuits” 


The results shown in Section 3.4 with regard to this circuit (Figure 3.7), and the values of other currents 
and voltages in this circuit as well, have profound implications. 

Resistors connected in such a way that current from one must flow only into another—currents in all 
resistors connected this way have the same magnitude—are said to be connected in series. For the two 
series-connected resistors in the example, the voltage across one resistor equals the ratio of that 
resistor’s value and the sum of resistances times the voltage across the series combination. 
This concept is so pervasive it has a name: voltage divider. 


12This content is available online at http: //cnx.org/content/m10674/2.8/. 
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The input-output relationship for this system, found in this particular case by voltage divider, takes 
the form of a ratio of the output voltage to the input voltage. 


Vout Re 


vn Rit Re 


In this way, we express how the components used to build the system affect the input-output relationship. 
Because this analysis was made with ideal circuit elements, we might expect this relation to break down if 
the input amplitude is too high (Will the circuit survive if the input changes from 1 volt to one million volts?) 
or if the source’s frequency becomes too high. In any case, this important way of expressing input-output 
relationships—as a ratio of output to input—pervades circuit and system theory. 

The current 7; is the current flowing out of the voltage source. Because it equals i2, we have that the 
ratio of the source’s voltage to the current flowing out of it equals rs = R, + Rz. Consequently, from the 
viewpoint of the source, it appears to be attached to a single resistor having resistance R, + Ro. 


RESISTORS IN SERIES: The series combination of two resistors acts, as far as the voltage source is 
concerned, as a single resistor having a value equal to the sum of the two resistances. 


This result is the first of several equivalent circuit ideas: In many cases, a complicated circuit when viewed 
from its terminals (the two places to which you might attach a source) appears to be a single circuit element 
(at best) or a simple combination of elements at worst. Thus, the equivalent circuit for a series combination 
of resistors is a single resistor having a resistance equal to the sum of its component resistances. 


+ 
vin(*) > Rog! &=> vin( =) R1+Ro 


Figure 3.8: The resistor (on the right) is equivalent to the two resistors (on the left) and has a resistance 
equal to the sum of the resistances of the other two resistors. 


Thus, the circuit the voltage source “feels” (through the current drawn from it) is a single resistor having 
resistance Ry + Rg. Note that in making this equivalent circuit, the output voltage can no longer be defined: 
The output resistor labeled Ry no longer appears. Thus, this equivalence is made strictly from the voltage 
source’s viewpoint. 


+ 


+ + 
in( A) Ry Ro i AvP vq R2Svo 


Figure 3.9: A simple parallel circuit. 


One interesting simple circuit (Figure 3.9) has two resistors connected side-by-side, what we will term a 
parallel connection, rather than in series. Here, applying KVL reveals that all the voltages are identical: 
vy = v and vg = v. This result typifies parallel connections. To write the KCL equation, note that the top 
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node consists of the entire upper interconnection section. The KCL equation is ij, — 71 — 72 = 0. Using the 


v-i relations, we find that 
Ri 


tout = tin 
Ri, + Re 


Exercise 3.6 (Solution on p. 94.) 
Suppose that you replaced the current source in Figure 3.9 by a voltage source. How would iout be 
related to the source voltage? Based on this result, what purpose does this revised circuit have? 


This circuit highlights some important properties of parallel circuits. You can easily show that the parallel 
-1 
1 Rik 
combination of R; and Rp has the v-i relation of a resistor having resistance | — + — 22. 
Ry Rez Ri + Rez 
A shorthand notation for this quantity is (R, || R2). As the reciprocal of resistance is conductance, we can 
say that for a parallel combination of resistors, the equivalent conductance is the sum of the 


conductances. 


ea eet ee ee ee eee 
O O 
R7{Ro 
Ry Ro — 
R7{+Roa 
O 5 eg 4 O Maes) ease: eae me ee ees - 
Figure 3.10 


Similar to voltage divider (p. 38) for series resistances, we have current divider for parallel resistances. 
The current through a resistor in parallel with another is the ratio of the conductance of the first to the 


sum of the conductances. Thus, for the depicted circuit, ig = cagacas Expressed in terms of resistances, 
1 2 

current divider takes the form of the resistance of the other resistor divided by the sum of resistances: 
: Ri, 

ig = —=——— i. 

R, + Re 

i 
ig 


Ry Re 


Figure 3.11 
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Figure 3.12: The simple attenuator circuit (Figure 3.7) is attached to an oscilloscope’s input. The 


input-output relation for the above circuit without a load is: vout = 


Suppose we want to pass the output signal into a voltage measurement device, such as an oscilloscope or 
a voltmeter. In system-theory terms, we want to pass our circuit’s output to a sink. For most applications, 
we can represent these measurement devices as a resistor, with the current passing through it driving the 
measurement device through some type of display. In circuits, a sink is called a load; thus, we describe a 
system-theoretic sink as a load resistance Ry. Thus, we have a complete system built from a cascade of 
three systems: a source, a signal processing system (simple as it is), and a sink. 

We must analyze afresh how this revised circuit, shown in Figure 3.12, works. Rather than defining eight 
variables and solving for the current in the load resistor, let’s take a hint from other analysis (series rules 
(p. 38), parallel rules (p. 40)). Resistors Rp and Ry are in a parallel configuration: The voltages across 
each resistor are the same while the currents are not. Because the voltages are the same, we can find the 
current through each from their v-i relations: ig = “put and ip = gu . Considering the node where all three 
resistors join, KCL says that the sum of the three currents must equal zero. Said another way, the current 
entering the node through R; must equal the sum of the other two currents leaving the node. Therefore, 


i, = 1g +iz, which means that 71 = Vout (4 + az): 


Let Req denote the equivalent resistance of the parallel combination of Rz and Rz. Using R,’s v-i 
relation, the voltage across it is vy = Firvout | The KVL equation written around the leftmost loop has 
eq 


Vin = V1 + Vout; Substituting for v1, we find 


Ri 
in — Vou —+1 
° — (ze ) 


Vout _ Req 
Vin Ri + Req 


or 


Thus, we have the input-output relationship for our entire system having the form of voltage divider, 
but it does not equal the input-output relation of the circuit without the voltage measurement device. We 
can not measure voltages reliably unless the measurement device has little effect on what we are trying to 
measure. We should look more carefully to determine if any values for the load resistance would lessen its 
impact on the circuit. Comparing the input-output relations before and after, what we need is Req © Ro. 

-1 
As Req = (z + x) , the approximation would apply if = > as or Rg < Ry. This is the condition 
Ro Rr Re Rr 
we seek: 


VOLTAGE MEASUREMENT: Voltage measurement devices must have large resistances compared with 
that of the resistor across which the voltage is to be measured. 


Exercise 3.7 (Solution on p. 94.) 
Let’s be more precise: How much larger would a load resistance need to be to affect the input-output 
relation by less than 10%? by less than 1%? 
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Example 3.1 


Rex R3 


_— 


Figure 3.13 


We want to find the total resistance of the example circuit. To apply the series and parallel 
combination rules, it is best to first determine the circuit’s structure: What is in series with what 
and what is in parallel with what at both small- and large-scale views. We have Rp in parallel 
with R3; this combination is in series with R4. This series combination is in parallel with R;. Note 
that in determining this structure, we started away from the terminals, and worked toward them. 
In most cases, this approach works well; try it first. The total resistance expression mimics the 
structure: 


Rr = R, || (Re || Rg + Ra) 


_ R, RoR3 + Ry RoR, + Ri R3R, 
~ Ry Ro + Ri R3 + RoR3 + RoR, + RR 


Such complicated expressions typify circuit “simplifications.” A simple check for accuracy is the 


units: Each component of the numerator should have the same units (here 93) as well as in the 
denominator (7). The entire expression is to have units of resistance; thus, the ratio of the 
numerator’s and denominator’s units should be ohms. Checking units does not guarantee accuracy, 
but can catch many errors. 


Another valuable lesson emerges from this example concerning the difference between cascading systems and 
cascading circuits. In system theory, systems can be cascaded without changing the input-output relation 
of intermediate systems. In cascading circuits, this ideal is rarely true unless the circuits are so designed. 
Design is in the hands of the engineer; he or she must recognize what have come to be known as loading 
effects. In our simple circuit, you might think that making the resistance Ry large enough would do the trick. 
Because the resistors R; and Rz can have virtually any value, you can never make the resistance of your 
voltage measurement device big enough. Said another way, a circuit cannot be designed in isolation 
that will work in cascade with all other circuits. Electrical engineers deal with this situation through 
the notion of specifications: Under what conditions will the circuit perform as designed? Thus, you will 
find that oscilloscopes and voltmeters have their internal resistances clearly stated, enabling you to determine 
whether the voltage you measure closely equals what was present before they were attached to your circuit. 
Furthermore, since our resistor circuit functions as an attenuator, with the attenuation (a fancy word for 


“1 
gains less than one) depending only on the ratio of the two resistor values Ee = (1 fz) , We can 


select any values for the two resistances we want to achieve the desired attenuation. The designer of this 
circuit must thus specify not only what the attenuation is, but also the resistance values employed so that 
integrators—people who put systems together from component systems—can combine systems together and 
have a chance of the combination working. 

Figure 3.14 summarizes the series and parallel combination results. These results are easy to remember 
and very useful. Keep in mind that for series combinations, voltage and resistance are the key quantities, 
while for parallel combinations current and conductance are more important. In series combinations, the 
currents through each element are the same; in parallel ones, the voltages are the same. 
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Figure 3.14: Series and parallel combination rules. (a) Rr = >> Rn, vn = Ftv (b) Gr = 3D Gn, 
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Exercise 3.8 (Solution on p. 94.) 


Contrast a series combination of resistors with a parallel one. Which variable (voltage or current) 
is the same for each and which differs? What are the equivalent resistances? When resistors are 
placed in series, is the equivalent resistance bigger, in between, or smaller than the component 
resistances? What is this relationship for a parallel combination? 


3.7 Equivalent Circuits: Resistors and Sources” 


We have found that the way to think about circuits is to locate and group parallel and series resistor 
combinations. Those resistors not involved with variables of interest can be collapsed into a single resistance. 
This result is known as an equivalent circuit: from the viewpoint of a pair of terminals, a group of resistors 
functions as a single resistor, the resistance of which can usually be found by applying the parallel and series 
rules. 


~_ 
= 
¢ 
a 
ye) 
< 


Figure 3.15 


This result generalizes to include sources in a very interesting and useful way. Let’s consider our simple 
attenuator circuit (shown in Figure 3.15) from the viewpoint of the output terminals. We want to find 
the v-i relation for the output terminal pair, and then find the equivalent circuit for the boxed circuit. To 
perform this calculation, use the circuit laws and element relations, but do not attach anything to the output 
terminals. We seek the relation between v and 7 that describes the kind of element that lurks within the 


dashed box. The result is 
Ro 


Dp p vin 
Ri + Re 


13This content is available online at http: //cnx.org/content/m0020/2.23/. 


v= (Ri | Ro) a+ (3.4) 
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If the source were zero, it could be replaced by a short circuit, which would confirm that the circuit does 
indeed function as a parallel combination of resistors. However, the source’s presence means that the circuit 
is not well modeled as a resistor. 


<O+ 


vea(*) 


Figure 3.16: The Thévenin equivalent circuit. 


If we consider the simple circuit of Figure 3.16, we find it has the v-i relation at its terminals of 


v = Regi + Veq (3.5) 
Comparing the two v-i relations, we find that they have the same form. In this case the Thévenin equiva- 
R 
lent resistance is R.q = (Ri || Ro) and the Thévenin equivalent source has voltage veq = RR 
1 2 


Thus, from viewpoint of the terminals, you cannot distinguish the two circuits. Because the equivalent 
circuit has fewer elements, it is easier to analyze and understand than any other alternative. 
For any circuit containing resistors and sources, the v-i relation will be of the form 


v = Regi + Ve 3.6 
q q 


and the Thévenin equivalent circuit for any such circuit is that of Figure 3.16. This equivalence applies no 
matter how many sources or resistors may be present in the circuit. In the example (Example 3.2) below, we 
know the circuit’s construction and element values, and derive the equivalent source and resistance. Because 
Thévenin’s theorem applies in general, we should be able to make measurements or calculations only from 
the terminals to determine the equivalent circuit. 

To be more specific, consider the equivalent circuit of Figure 3.16. Let the terminals be open-circuited, 
which has the effect of setting the current i to zero. Because no current flows through the resistor, the voltage 
across it is zero (remember, Ohm’s Law says that v = Ri). Consequently, by applying KVL we have that 
the so-called open-circuit voltage voc equals the Thévenin equivalent voltage. Now consider the situation 
when we set the terminal voltage to zero (short-circuit it) and measure the resulting current. Referring to 
the equivalent circuit, the source voltage now appears entirely across the resistor, leaving the short-circuit 


Uv : F 
current to be ig. = -—“*. From this property, we can determine the equivalent resistance. 
eq 
Yeq = Voc (3.7) 
v 
Req = -—— (3.8) 
Use 
Exercise 3.9 (Solution on p. 94.) 


Use the open/short-circuit approach to derive the Thévenin equivalent of the circuit shown in 
Figure 3.17. 
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Figure 3.17 
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Figure 3.18 


For the circuit depicted in Figure 3.18, let’s derive its Thévenin equivalent two different ways. 

Starting with the open/short-circuit approach, let’s first find the open-circuit voltage vo... We have 

a current divider relationship as R, is in parallel with the series combination of R2 and R3. Thus, 
R3R, 


Coe => = 
oc R, + Ro+ Rg 
thus no current flows through it. In short, R3 does not affect the short-circuit current, and can be 


iin When we short-circuit the terminals, no voltage appears across R3, and 


R 
eliminated. We again have a current divider relationship: ig. = ay etn Thus, the Thévenin 
1 2 
equivalent resistance is Aig (i ib) 
R, + Ro4+ Rg ; 


To verify, let’s find the equivalent resistance by reaching inside the circuit and setting the current 
source to zero. Because the current is now zero, we can replace the current source by an open circuit. 
From the viewpoint of the terminals, resistor R3 is now in parallel with the series combination of 
R, and Ry. Thus, Req = (R3 || Ri + Re), and we obtain the same result. 

As you might expect, equivalent circuits come in two forms: the voltage-source oriented Thévenin equiva- 


lent!4 and the current-source oriented Mayer-Norton equivalent (Figure 3.19). To derive the latter, the 
v-i relation for the Thévenin equivalent can be written as 


Vv = Regt + Veq (3.9) 
or ‘ 

i= a — leq (3.10) 
where teg = ved ig the Mayer-Norton equivalent source. The Mayer-Norton equivalent shown in Figure 3.19 


eq 
can be easily shown to have this v-i relation. Note that both variations have the same equivalent resistance. 
The short-circuit current equals the negative of the Mayer-Norton equivalent source. 


14“Finding Thévenin Equivalent Circuits” http: //cnx.org/content/m0021/latest/ 
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Figure 3.19: All circuits containing sources and resistors can be described by simpler equivalent circuits. 
Choosing the one to use depends on the application, not on what is actually inside the circuit. 


Exercise 3.10 (Solution on p. 94.) 
Find the Mayer-Norton equivalent circuit for the circuit below. 
C 
R2 
iin Ri Rg 
C 
Figure 3.20 


Equivalent circuits can be used in two basic ways. The first is to simplify the analysis of a complicated 
circuit by realizing the any portion of a circuit can be described by either a Thévenin or Mayer-Norton 
equivalent. Which one is used depends on whether what is attached to the terminals is a series configuration 
(making the Thévenin equivalent the best) or a parallel one (making Mayer-Norton the best). 

Another application is modeling. When we buy a flashlight battery, either equivalent circuit can accu- 
rately describe it. These models help us understand the limitations of a battery. Since batteries are labeled 
with a voltage specification, they should serve as voltage sources and the Thévenin equivalent serves as the 
natural choice. If a load resistance Ry is placed across its terminals, the voltage output can be found using 

Rr 
ey ie 
resistance, then, to a good approximation, the battery does serve as a voltage source. If the load resistance 
is much smaller, we certainly don’t have a voltage source (the output voltage depends directly on the load 
resistance). Consider now the Mayer-Norton equivalent; the current through the load resistance is given by 

Req 
Ryr+ Req 
this resistance should be much smaller than the equivalent resistance. If the load resistance is comparable 
to the equivalent resistance, the battery serves neither as a voltage source or a current course. Thus, when 


voltage divider: v = If we have a load resistance much larger than the battery’s equivalent 


current divider, and equals 1 = — ) ieq. For a current that does not vary with the load resistance, 
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you buy a battery, you get a voltage source if its equivalent resistance is much smaller than the equivalent 
resistance of the circuit to which you attach it. On the other hand, if you attach it to a circuit having a 
small equivalent resistance, you bought a current source. 


LEON CHARLES THEVENIN: He was an engineer with France’s Postes, Télégraphe et Téléphone. In 
1883, he published (twice!) a proof of what is now called the Thévenin equivalent while developing 
ways of teaching electrical engineering concepts at the Ecole Polytechnique. He did not realize that 
the same result had been published by Hermann Helmholtz,!° the renowned nineteenth century 
physicist, thirty years earlier. 


HANS FERDINAND Mayer: After earning his doctorate in physics in 1920, he turned to com- 
munications engineering when he joined Siemens & Halske in 1922. In 1926, he published in a 
German technical journal the Mayer-Norton equivalent. During his interesting career, he rose to 
lead Siemens’s Central Laboratory in 1936, surreptitiously leaked to the British all he knew of 
German warfare capabilities a month after the Nazis invaded Poland, was arrested by the Gestapo 
in 1943 for listening to BBC radio broadcasts, spent two years in Nazi concentration camps, and 
went to the United States for four years working for the Air Force and Cornell University before 
returning to Siemens in 1950. He rose to a position on Siemens’s Board of Directors before retiring. 


Epwarp L. Norton: Edward Norton!® was an electrical engineer who worked at Bell Laboratory 
from its inception in 1922. In the same month when Mayer’s paper appeared, Norton wrote in an 
internal technical memorandum a paragraph describing the current-source equivalent. No evidence 
suggests Norton knew of Mayer’s publication. 


3.8 Circuits with Capacitors and Inductors” 


Let’s consider a circuit having something other than resistors and sources, such as shown in Figure 3.21. 
Because of KVL, we know that vin = UR + Vout. The current through the capacitor is given by i = CL vouts 
and this current equals that passing through the resistor. Substituting ve = Ri into the KVL equation and 
using the v-i relation for the capacitor, we arrive at 


d 
RC] Vout + Vout = Vin (3.11) 


The input-output relation for circuits involving energy storage elements takes the form of an ordinary differ- 
ential equation, which we must solve to determine what the output voltage is for a given input. In contrast 
to resistive circuits, where we obtain an explicit input-output relation, we now have an implicit relation 
that requires more work to obtain answers. 

At this point, we could learn how to solve differential equations. Note first that even finding the differential 
equation relating an output variable to a source is often very tedious. The parallel and series combination 
rules that apply to resistors don’t directly apply when capacitors and inductors occur. We would have to slog 
our way through the circuit equations, simplifying them until we finally found the equation that related the 
source(s) to the output. At the turn of the twentieth century, a method was discovered that not only made 
finding the differential equation easy, but also simplified the solution process in the most common situation. 
Although not original with him, Charles Steinmetz!® presented the key paper describing the impedance 
approach in 1893. It allows circuits containing capacitors and inductors to be solved with the same methods 
we have learned to solved resistor circuits. To use impedances, we must master complex numbers. Though 
the arithmetic of complex numbers is mathematically more complicated than with real numbers, the increased 
insight into circuit behavior and the ease with which circuits are solved with impedances is well worth the 
diversion. But more importantly, the impedance concept is central to engineering and physics, having a 
reach far beyond just circuits. 


http: //www-gap.dcs.st-and.ac.uk/~history/Biographies/Helmholtz. html 
16http://www.ece.rice.edu/~dhj/norton 

17This content is available online at http: //cnx.org/content/m0023/2.12/. 
18nttp: //www.edisontechcenter.org/CharlesProteusSteinmetz.html 
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Figure 3.21: A simple RC circuit. 


i F iV | 
R Vv C==v L Sy 


(a) (b) (c) 


1 


Figure 3.22: (a) Resistor: Zr = R (b) Capacitor: Zc = janfC 


(c) Inductor: Zr, = j2nfL 


3.9 The Impedance Concept” 


Rather than solving the differential equation that arises in circuits containing capacitors and inductors, let’s 
pretend that all sources in the circuit are complex exponentials having the same frequency. Although this 
pretense can only be mathematically true, this fiction will greatly ease solving the circuit no matter what 
the source really is. 

For the above example RC circuit (Figure 3.21), let vin = Vine??"/*. The complex amplitude Vj, deter- 
mines the size of the source and its phase. The critical consequence of assuming that sources have this form 
is that all voltages and currents in the circuit are also complex exponentials, having amplitudes governed 
by KVL, KCL, and the v-i relations and the same frequency as the source. To appreciate why this should 
be true, let’s investigate how each circuit element behaves when either the voltage or current is a complex 


exponential. For the resistor, v = Ri. When v = Ve/?™/*; then i = Ro Thus, if the resistor’s voltage is 


a complex exponential, so is the current, with an amplitude J = — (determined by the resistor’s v-i relation) 


and a frequency the same as the voltage. Clearly, if the current were assumed to be a complex exponential, 
so would the voltage. For a capacitor, i = C’ 4 (v). Letting the voltage be a complex exponential, we have 
i = CVj2nfei?*/*. The amplitude of this complex exponential is J = CVj2rf. Finally, for the inductor, 
where v = L4 (i), assuming the current to be a complex exponential results in the voltage having the form 
v = LIj2nfei?"/*, making its complex amplitude V = LIj27f. 

The major consequence of assuming complex exponential voltage and currents is that the 


V 
ratio Z = — for each element does not depend on time, but does depend on source frequency. 


This quantity is known as the element’s impedance. 

The impedance is, in general, a complex-valued, frequency-dependent quantity. For example, the magni- 
tude of the capacitor’s impedance is inversely related to frequency, and has a phase of —7/2. This observation 
means that if the current is a complex exponential and has constant amplitude, the amplitude of the voltage 
decreases with frequency. 

Let’s consider Kirchhoff’s circuit laws. When voltages around a loop are all complex exponentials of the 


19This content is available online at http: //cnx.org/content/m0024/2.23/. 
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same frequency, we have 


Se Le 0 (3.12) 


n 


which means 


Se (3.13) 


the complex amplitudes of the voltages obey KVL. We can easily imagine that the complex amplitudes 
of the currents obey KCL. 

What we have discovered is that source(s) equaling a complex exponential of the same frequency forces 
all circuit variables to be complex exponentials of the same frequency. Consequently, the ratio of voltage to 
current for each element equals the ratio of their complex amplitudes, which depends only on the source’s 
frequency and element values. 

This situation occurs because the circuit elements are linear and time-invariant. For example, suppose we 
had a circuit element where the voltage equaled the square of the current: v(t) = Ki? (t). If i (t) = Ie??"/*, 
v(t) = KI’e/?7?Ft, meaning that voltage and current no longer had the same frequency and that their ratio 
was time-dependent. 

Because for linear circuit elements the complex amplitude of voltage is proportional to the complex 
amplitude of current— V = ZI — assuming complex exponential sources means circuit elements behave 
as if they were resistors, where instead of resistance, we use impedance. Because complex amplitudes 
for voltage and current also obey Kirchhoff’s laws, we can solve circuits using voltage and 
current divider and the series and parallel combination rules by considering the elements to 
be impedances. 


3.10 Time and Frequency Domains” 


When we find the differential equation relating the source and the output, we are faced with solving the 
circuit in what is known as the time domain. What we emphasize here is that it is often easier to find 
the output if we use impedances. Because impedances depend only on frequency, we find ourselves in the 
frequency domain. A common error in using impedances is keeping the time-dependent part, the complex 
exponential, in the fray. The entire point of using impedances is to get rid of time and concentrate on 
frequency. Only after we find the result in the frequency domain do we go back to the time domain and put 
things back together again. 

To illustrate how the time domain, the frequency domain and impedances fit together, consider the time 
domain and frequency domain to be two work rooms. Since you can’t be two places at the same time, you 
are faced with solving your circuit problem in one of the two rooms at any point in time. Impedances and 
complex exponentials are the way you get between the two rooms. Security guards make sure you don’t try 
to sneak time domain variables into the frequency domain room and vice versa. Figure 3.23 shows how this 
works. 


20This content is available online at http: //cnx.org/content/m10708/2.9/. 
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Vout(t) = --- Vout = VintH(f) 


Figure 3.23: The time and frequency domains are linked by assuming signals are complex exponentials. 
In the time domain, signals can have any form. Passing into the frequency domain “work room,” signals 
are represented entirely by complex amplitudes. 


As we unfold the impedance story, we’ll see that the powerful use of impedances suggested by Steinmetz?! 
greatly simplifies solving circuits, alleviates us from solving differential equations, and suggests a general way 
of thinking about circuits. Because of the importance of this approach, let’s go over how it works. 


1. Even though it’s not, pretend the source is a complex exponential. We do this because the impedance 
approach simplifies finding how input and output are related. If it were a voltage source having voltage 
Vin = p(t) (a pulse), still let vin = VineJ?7/*. We'll learn how to “get the pulse back” later. 

2. With a source equaling a complex exponential, all variables in a linear circuit will also be complex 
exponentials having the same frequency. The circuit’s only remaining “mystery” is what each variable’s 
complex amplitude might be. To find these, we consider the source to be a complex number (Vin here) 
and the elements to be impedances. 

3. We can now solve using series and parallel combination rules how the complex amplitude of any variable 
relates to the sources complex amplitude. 


Example 3.3 


To illustrate the impedance approach, we refer to the RC circuit shown in Figure 3.24, and we 
assume that vin = Vine?27/*. 


2lnttp://www.edisontechcenter.org/CharlesProteusSteinmetz. html 


51 


+ ae + 
Vin C) C== Vout Vin(*) Bal Wy 
a t 
_ = C Ou 


(a) (b) 


Figure 3.24: (a) A simple RC circuit. (b) The impedance counterpart for the RC circuit. Note that 
the source and output voltage are now complex amplitudes. 


Using impedances, the complex amplitude of the output voltage Vj, can be found using voltage 
divider: 
Zc 
Vout = Zot in i™ 
1 
j2anfC 
= V; 
ia in 
j2nfC +R 
1 


ey 
janfRC+1 


Referring to the differential equation for this circuit (shown in Section 3.8, page 47, to be 
RC £ Vout + Vout = Vin), when we let the output and input voltages be complex exponentials, we obtain 
the same relationship between their complex amplitudes. Thus, using impedances is equivalent to using the 
differential equation and solving it when the source(s) is a complex exponential. 

In fact, we can find the differential equation directly using impedances. If we cross-multiply the relation 
between input and output amplitudes, 


Vout (G2af RC +1) = Vin 
and then put the complex exponentials back in, we have 
RC§j2m fVourer" + Voure?7 st = Vine? ?™ 
In the process of defining impedances, note that the factor j727f arises from the derivative of a complex 
exponential. We can reverse the impedance process, and revert back to the differential equation. 
d 
ROT Vout + Vout = Vin 


This is the same equation that was derived much more tediously in Section 3.8. Finding the differential 

equation relating output to input is far simpler when we use impedances than with any other technique. 
Exercise 3.11 (Solution on p. 94.) 
Suppose you had an expression where a complex amplitude was divided by 727 f. What time-domain 
operation corresponds to this division? 


3.11 Power in the Frequency Domain” 


Recalling that the instantaneous power consumed by a circuit element or an equivalent circuit that represents 
a collection of elements equals the voltage times the current entering the positive-voltage terminal, p(t) = 


?2This content is available online at http: //cnx.org/content/m17308/1.2/. 
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vu (t)i(t), what is the equivalent expression using impedances? The resulting calculation reveals more about 
power consumption in circuits and the introduction of the concept of average power. 

When all sources produce sinusoids of frequency f, the voltage and current for any circuit element or 
collection of elements are sinusoids of the same frequency. 


vu (t) = |V| cos (27 ft + 6) 
i(t) = |I| cos (27 ft + 0) 


Here, the complex amplitude of the voltage V equals |V|e/? and that of the current is |I|e?®. We can also 
write the voltage and current in terms of their complex amplitudes using Euler’s formula (Section 2.1.2). 


1 igs i egne 
(Ve pyre ent) 
. 1 ic ee 
oO), leita sn) 
Multiplying these two expressions and simplifying gives 
1 ) . 
Pp (t) = 4 (VI + Vit + VIci4nft + Vier) 


1 1 a 
= 5Re[VI"] + 5Re [VIe"*"""] 


1 1 
= 5Re[VI"] + 5|V||Z| cos (An ft + 6 +8) 


We define sVI * to be complex power. The real-part of complex power is the first term and since it does 
not change with time, it represents the power consistently consumed/produced by the circuit. The second 
term varies with time at a frequency twice that of the source. Conceptually, this term details how power 
“sloshes” back and forth in the circuit because of the sinusoidal source. 

From another viewpoint, the real-part of complex power represents long-term energy consump- 
tion/production. Energy is the integral of power and, as the integration interval increases, the first term 
appreciates while the time-varying term “sloshes.” Consequently, the most convenient definition of the av- 
erage power consumed /produced by any circuit is in terms of complex amplitudes. 


1 
Pave = Re [VI"] (3.14) 


Exercise 3.12 (Solution on p. 94.) 
Suppose the complex amplitudes of the voltage and current have fixed magnitudes. What phase 
relationship between voltage and current maximizes the average power? In other words, how are ¢ 
and @ related for maximum power dissipation? 


Because the complex amplitudes of the voltage and current are related by the equivalent impedance, average 
power can also be written as 
1 2 1 1 2 
Paye = =Re |Z] |I|" = =Re | =] |V 
wo = 5Re(Z] TP = 5Re || IV 
These expressions generalize the results (3.3) we obtained for resistor circuits. We have derived a fundamental 
result: Only the real part of impedance contributes to long-term power dissipation. Of the circuit 
elements, only the resistor dissipates power. Capacitors and inductors dissipate no power in the long term. 
It is important to realize that these statements apply only for sinusoidal sources. If you turn on a constant 
voltage source in an RC-circuit, charging the capacitor does consume power. 
Exercise 3.13 (Solution on p. 95.) 
In an earlier problem (Problem 1.1), we found that the rms value of a sinusoid was its amplitude 
divided by /2. What is average power expressed in terms of the rms values of the voltage and 
current (Vims and [pms respectively)? 
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(b) Equivalent circuits with impedances. 


Figure 3.25: Comparing the first, simpler, figure with the slightly more complicated second figure, we 
see two differences. First of all, more circuits (all those containing linear elements in fact) have equivalent 
circuits that contain equivalents. Secondly, the terminal and source variables are now complex amplitudes, 
which carries the implicit assumption that the voltages and currents are single complex exponentials, all 
having the same frequency. 


3.12 Equivalent Circuits: Impedances and Sources” 


When we have circuits with capacitors and/or inductors as well as resistors and sources, Thévenin and Mayer- 
Norton equivalent circuits can still be defined by using impedances and complex amplitudes for voltage and 
currents. For any circuit containing sources, resistors, capacitors, and inductors, the input-output relation 
for the complex amplitudes of the terminal voltage and current is 


V = Zegl + Vea 
V 
f= = 1, 

Ze 


with Veg = Zeqleq. Thus, we have Thévenin and Mayer-Norton equivalent circuits as shown in Figure 3.25. 
Example 3.4 
Let’s find the Thévenin and Mayer-Norton equivalent circuits for the circuit shown in Figure 3.26. 
The open-circuit voltage and short-circuit current techniques still work, except we use impedances 


23This content is available online at http: //cnx.org/content/m0030/2.20/. 
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Figure 3.26 


and complex amplitudes. The open-circuit voltage corresponds to the transfer function we have 
already found. When we short the terminals, the capacitor no longer has any effect on the circuit, 


out 


and the short-circuit current J;. equals . The equivalent impedance can be found by setting 
the source to zero, and finding the impedance using series and parallel combination rules. In 


our case, the resistor and capacitor are in parallel once the voltage source is removed (setting it 


to zero amounts to replacing it with a short-circuit). Thus, Zeq = Ril aa er + j2nfRO’ 
Consequently, we have 


1 
Veg = a pA Vin 
IT 1+ 92rfRC 
1 
I, = =Vin 
Ue 
R 


Zeq = 79 FA 
tT 1+ 92rfRC 
Again, we should check the units of our answer. Note in particular that 727 fRC must be dimen- 
sionless. Is it? 


3.13 Transfer Functions” 


The ratio of the output and input amplitudes for Figure 3.27, known as the transfer function or the 


frequency response, is given by 
Vout 


Vin 


= H(f) 
1 

— janfRC+1 
Implicit in using the transfer function is that the input is a complex exponential, and the output is also a 
complex exponential having the same frequency. The transfer function reveals how the circuit modifies the 
input amplitude in creating the output amplitude. Thus, the transfer function completely describes how 
the circuit processes the input complex exponential to produce the output complex exponential. The circuit’s 
function is thus summarized by the transfer function. In fact, circuits are often designed to meet transfer 
function specifications. Because transfer functions are complex-valued, frequency-dependent quantities, we 
can better appreciate a circuit’s function by examining the magnitude and phase of its transfer function 
(Figure 3.28). 

This transfer function has many important properties and provides all the insights needed to determine 
how the circuit functions. First of all, note that we can compute the frequency response for both positive and 
negative frequencies. Recall that sinusoids consist of the sum of two complex exponentials, one having the 
negative frequency of the other. We will consider how the circuit acts on a sinusoid soon. Do note that the 


(3.15) 


24This content is available online at http: //cnx.org/content/m0028/2.19/. 
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Figure 3.27: A simple RC circuit. 


Figure 3.28: Magnitude and phase of the transfer function of the RC circuit shown in Figure 3.27 when 
RC =1. (a) |A(f)|= (b) ZA (f) = — arctan (2a f RC) 


V(2nfRC)? +1 


magnitude has even symmetry: The negative frequency portion is a mirror image of the positive frequency 
portion: |H (—f)| = |H (f)|. The phase has odd symmetry: ZH (—f) = —ZH(f). These properties of 
this specific example apply for all transfer functions associated with circuits. Consequently, we don’t need 
to plot the negative frequency component; we know what it is from the positive frequency part. 

The magnitude equals 1/\/2 of its maximum gain (one at f = 0) when 27fRC = 1 (the two terms in 


the denominator of the magnitude are equal). The frequency f. = defines the boundary between two 


2nRC 
operating ranges. 
e For frequencies below this frequency, the circuit does not much alter the amplitude of the complex 
exponential source. 
e For frequencies greater than f,, the circuit strongly attenuates the amplitude. Thus, when the source 
frequency is in this range, the circuit’s output has a much smaller amplitude than that of the source. 


For these reasons, this frequency is known as the cutoff frequency. In this circuit the cutoff frequency 
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depends only on the product of the resistance and the capacitance. Thus, a cutoff frequency of 1 kHz occurs 


-3 
= 10° or RC = " 
100 nF Or 10 2 and 1.59 uF result in the same cutoff frequency. 

The phase shift caused by the circuit at the cutoff frequency precisely equals —7/4. Thus, below the 
cutoff frequency, phase is little affected, but at higher frequencies, the phase shift caused by the circuit 
becomes —7/2. This phase shift corresponds to the difference between a cosine and a sine. 

We can use the transfer function to find the output when the input voltage is a sinusoid for two reasons. 
First of all, a sinusoid is the sum of two complex exponentials, each having a frequency equal to the negative 
of the other. Secondly, because the circuit is linear, superposition applies. If the source is a sine wave, we 
know that 


when = 1.59 x 107+. Thus resistance-capacitance combinations of 1.59 kQ and 


Vin (t) = Asin (27 ft) 
A 
2j 
Since the input is the sum of two complex exponentials, we know that the output is also a sum of two similar 
complex exponentials, the only difference being that the complex amplitude of each is multiplied by the 
transfer function evaluated at each exponential’s frequency. 


(3.16) 


(ef2" ft oa erat?) 


vout (t) = AH (pest — An (—py ett (3.17) 


As noted earlier, the transfer function is most conveniently expressed in polar form: H (f) = |H (f) |e?<#™. 
Furthermore, |H (—f)| = |H (f)| (even symmetry of the magnitude) and ZH (—f) = —ZH (f) (odd sym- 
metry of the phase). The output voltage expression simplifies to 


A A 
out (t) = —|H Qn ft+ZH(f)) __ “| —j2nft+ZH(f)) 
vont t) = Sol (A) Ie si (F) oe 


= A|H (f)|sin (Qn ft+ ZH (f)) 


The circuit’s output to a sinusoidal input is also a sinusoid, having a gain equal to the magnitude 
of the circuit’s transfer function evaluated at the source frequency and a phase equal to the 
phase of the transfer function at the source frequency. It will turn out that this input-output relation 
description applies to any linear circuit having a sinusoidal source. 


Exercise 3.14 (Solution on p. 95.) 
This input-output property is a special case of a more general result. Show that if the source can 
be written as the imaginary part of a complex exponential— vy, (t) = Im [Ve!?"/*] — the output 
is given by Vout (t) = Im [VH (f) e??*/*]. Show that a similar result also holds for the real part. 


The notion of impedance arises when we assume the sources are complex exponentials. This assumption 
may seem restrictive; what would we do if the source were a unit step? When we use impedances to find the 
transfer function between the source and the output variable, we can derive from it the differential equation 
that relates input and output. The differential equation applies no matter what the source may be. As 
we have argued, it is far simpler to use impedances to find the differential equation (because we can use 
series and parallel combination rules) than any other method. In this sense, we have not lost anything by 
temporarily pretending the source is a complex exponential. 

In fact we can also solve the differential equation using impedances! Thus, despite the apparent restric- 
tiveness of impedances, assuming complex exponential sources is actually quite general. 


3.14 Designing Transfer Functions” 


If the source consists of two (or more) signals, we know from linear system theory that the output voltage 
equals the sum of the outputs produced by each signal alone. In short, linear circuits are a special case 


25This content is available online at http: //cnx.org/content/m0031/2.21/. 
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Figure 3.29 


of linear systems, and therefore superposition applies. In particular, suppose these component signals are 
complex exponentials, each of which has a frequency different from the others. The transfer function portrays 
how the circuit affects the amplitude and phase of each component, allowing us to understand how the circuit 
works on a complicated signal. Those components having a frequency less than the cutoff frequency pass 
through the circuit with little modification while those having higher frequencies are suppressed. The circuit 
is said to act as a filter, filtering the source signal based on the frequency of each component complex 
exponential. Because low frequencies pass through the filter, we call it a lowpass filter to express more 
precisely its function. 

We have also found the ease of calculating the output for sinusoidal inputs through the use of the transfer 
function. Once we find the transfer function, we can write the output directly as indicated by the output of 
a circuit for a sinusoidal input (3.18). 


Example 3.5 

Let’s apply these results to a final example, in which the input is a voltage source and the output 
is the inductor current. The source voltage equals Vi, = 2cos (2760t) + 3. We want the circuit to 
pass constant (offset) voltage essentially unaltered (save for the fact that the output is a current 
rather than a voltage) and remove the 60 Hz term. Because the input is the sum of two sinusoids—a 
constant is a zero-frequency cosine—-our approach is 


find the transfer function using impedances; 

use it to find the output due to each input component; 
add the results; 

find element values that accomplish our design criteria. 


aac 


Because the circuit is a series combination of elements, let’s use voltage divider to find the transfer 
function between V;, and V, then use the v-i relation of the inductor to find its current. 


Tout G20 f L 1 
Vn R+j2arflL j2nfLl 
ee see (3.19) 
janflL+R 
=y) 
where Sci 
voltage divider = ae 
and 


inductor admittance = 


jar fl 


[Do the units check?] The form of this transfer function should be familiar; it is a lowpass filter, 
and it will perform our desired function once we choose element values properly. 
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Figure 3.30: Input and output waveforms for the example RL circuit when the element values are 
R= 6.28Q and L = 100mH. 


3 
The constant term is easiest to handle. The output is given by 3|H (0)| = —. Thus, the value we 


choose for the resistance will determine the scaling factor of how voltage is converted into current. 
For the 60 Hz component signal, the output current is 2|H (60) | cos (2760t + 7H (60)). The total 
output due to our source is 


tout = 2|H (60) | cos (2760t + ZH (60)) + 3A (0) (3.20) 
The cutoff frequency for this filter occurs when the real and imaginary parts of the transfer function’s 
denominator equal each other. Thus, 27f.L = R, which gives f. = ——. We want this cutoff 
frequency to be much less than 60 Hz. Suppose we place it at, say, 10 Hz. This specification would 
require the component values to be related by ria 207 = 62.8. The transfer function at 60 Hz 


would be 


j2760L + R R}67 +1 RVJ37 R 
which yields an attenuation (relative to the gain at zero frequency) of about 1/6, and result in 
an output amplitude of 0.3/R relative to the constant term’s amplitude of 3/R. A factor of ten 
difference between the relative sizes of the two components seems reasonable. Having a 100 mH 
inductor would require a 6.28 2 resistor. An easily available resistor value is 6.8 Q; thus, this 
choice results in cheaply and easily purchased parts. To make the resistance bigger would require a 
proportionally larger inductor. Unfortunately, even a 1 H inductor is physically large; consequently 
low cutoff frequencies require small-valued resistors and large-valued inductors. The choice made 
here represents only one compromise. 

The phase of the 60 Hz component will very nearly be —7/2, leaving it to be 
0.3/R cos (2760t — ) = 0.3/Rsin (2760t). The waveforms for the input and output are shown 
in Figure 3.30. 


Note that the sinusoid’s phase has indeed shifted; the lowpass filter not only reduced the 60 Hz signal’s 
amplitude, but also shifted its phase by 90°. 
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Figure 3.31 


3.15 Formal Circuit Methods: Node Method” 


In some (complicated) cases, we cannot use the simplification techniques—such as parallel or series combi- 
nation rules—to solve for a circuit’s input-output relation. In other modules, we wrote v-i relations and 
Kirchhoff’s laws haphazardly, solving them more on intuition than procedure. We need a formal method 
that produces a small, easy set of equations that lead directly to the input-output relation we seek. One 
such technique is the node method. 

The node method begins by finding all nodes—places where circuit elements attach to each other—in the 
circuit. We call one of the nodes the reference node; the choice of reference node is arbitrary, but it is 
usually chosen to be a point of symmetry or the “bottom” node. For the remaining nodes, we define node 
voltages e,, that represent the voltage between the node and the reference. These node voltages constitute 
the only unknowns; all we need is a sufficient number of equations to solve for them. In our example, we 
have two node voltages. The very act of defining node voltages is equivalent to using all the KVL 
equations at your disposal. The reason for this simple, but astounding, fact is that a node voltage is 
uniquely defined regardless of what path is traced between the node and the reference. Because two paths 
between a node and reference have the same voltage, the sum of voltages around the loop equals zero. 

In some cases, a node voltage corresponds exactly to the voltage across a voltage source. In such cases, 
the node voltage is specified by the source and is not an unknown. For example, in our circuit, e; = Vin; 
thus, we need only to find one node voltage. 

The equations governing the node voltages are obtained by writing KCL equations at each node having 
an unknown node voltage, using the v-i relations for each element. In our example, the only circuit equation 
is 

€2 — Vin €2 €2 
Ry * Ro a R3 
A little reflection reveals that when writing the KCL equations for the sum of currents leaving a node, that 
node’s voltage will always appear with a plus sign, and all other node voltages with a minus sign. Systematic 
application of this procedure makes it easy to write node equations and to check them before solving them. 
Also remember to check units at this point: Every term should have units of current. In our example, solving 
for the unknown node voltage is easy: 


0 (3.22) 


_ R2R3 . 
~ RR. + RiR3 + RRs” 


Have we really solved the circuit with the node method? Along the way, we have used KVL, KCL, and 
the v-i relations. Previously, we indicated that the set of equations resulting from applying these laws is 
necessary and sufficient. This result guarantees that the node method can be used to “solve” any circuit. 
One fallout of this result is that we must be able to find any circuit variable given the node voltages and 
sources. All circuit variables can be found using the v-i relations and voltage divider. For example, the 


(3.23) 


€2 


current through R3 equals 
3 


26This content is available online at http: //cnx.org/content/m0032/2.20/. 


60 CHAPTER 3. ANALOG SIGNAL PROCESSING 


ey eo 


in( A) Ry R3 


Figure 3.32 


The presence of a current source in the circuit does not affect the node method greatly; just include it in 
writing KCL equations as a current leaving the node. The circuit has three nodes, requiring us to define 
two node voltages. The node equations are 


ey €1 — €2 
+ 


Ry Ro 


€g2—- €1 [) 
— =0 Node 2 
Rs Rs (Node 2) 


—tin =0 (Node 1) 


Note that the node voltage corresponding to the node that we are writing KCL for enters with a positive 
sign, the others with a negative sign, and that the units of each term is given in amperes. Rewrite these 
equations in the standard set-of-linear-equations form. 


Solving these equations gives 


: R2+ Rg : 
= Re 2 
Ri R3 
62> tin 
Ri, + Ro+ Rs 


To find the indicated current, we simply use i = -_ 
3 


Example 3.6: Node Method Example 

In the circuit shown in Figure 3.33, we cannot use the series/parallel combination rules: The vertical 
resistor at node 1 keeps the two horizontal 1 2 resistors from being in series, and the 2 () resistor 
prevents the two 1 (Q resistors at node 2 from being in series. We really do need the node method 
to solve this circuit! Despite having six elements, we need only define two node voltages. The node 
equations are 


= Node 1 
i i i 0 (Node 1) 
€2 — Vin €2 €2— €1 
= 0 Node 2 
2 1 1 ode) 
Solving these equations yields e; = ovin and €2 = Vin: The output current equals 2 = Vin: 


One unfortunate consequence of using the element’s numeric values from the outset is that it 
becomes impossible to check units while setting up and solving equations. 
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vin C) 1 


Figure 3.33 


Exercise 3.15 (Solution on p. 95.) 
What is the equivalent resistance seen by the voltage source in Figure 3.33? 


The node method applies to RLC circuits, without significant modification from the methods used on simple 
resistive circuits, if we use complex amplitudes. We rely on the fact that complex amplitudes satisfy KVL, 
KCL, and impedance-based v-i relations. In the example circuit shown in Figure 3.34, we define complex 
amplitudes for the input and output variables and for the node voltages. We need only one node voltage 
here, and its KCL equation is Pa - 
in . 
R, PEPTIC LT S =0 
with the result 
E= = 
~ Ry + Rot j2af Ri R2C 


Vin 


E 
To find the transfer function between input and output voltages, we compute the ratio —. The transfer 
function’s magnitude and angle are = 


R 
| (f)| = ; 
(Ri + Ro)? + 2rf Ri Rac)? 
ZH (f) = —arctan (ae) 
1 2 


This circuit differs from the one shown previously (Figure 3.27) in that the resistor Ry has been added across 
the output. What effect has it had on the transfer function, which in the original circuit was a lowpass filter 


1 
having cutoff frequency f, = aR a As shown in Figure 3.35, adding the second resistor has two effects: 
Ty 


it lowers the gain in the passband (the range of frequencies for which the filter has little effect on the input) 
and increases the cutoff frequency. 


Figure 3.34: Modification of the circuit shown on the left to illustrate the node method and the effect 
of adding the resistor Re. 


62 CHAPTER 3. ANALOG SIGNAL PROCESSING 


| Ry=1, Ro=1 


Ly L 1 L L {ie f 
0 1 1 Ry+Re 1 
2mRC 2nRyC° Ro 


Figure 3.35: Transfer functions of the circuits shown in Figure 3.34. Here, Ri = 1, Re =1, and C =1. 


When Rz = Rj, as shown on the plot, the passband gain becomes half of the original, and the cutoff 
frequency increases by the same factor. Thus, adding R2 provides a ’knob’ by which we can trade passband 
gain for cutoff frequency. 


Exercise 3.16 (Solution on p. 95.) 
We can change the cutoff frequency without affecting passband gain by changing the resistance in 
the original circuit. Does the addition of the R»2 resistor help in circuit design? 


3.16 Power Conservation in Circuits” 


Now that we have a formal method—the node method—for solving circuits, we can use it to prove a powerful 
result: KVL and KCL are all that are required to show that all circuits conserve power, regardless of what 
elements are used to build the circuit. 

First of all, define node voltages for all nodes in a given circuit. Any node chosen as the reference will 
do. For example, in the portion of a large circuit depicted in Figure 3.36, we define node voltages for nodes 
a, b and c. With these node voltages, we can express the voltage across any element in terms of them. For 
example, the voltage across element 1 is given by v1 = ep — ea. The instantaneous power for element 1 
becomes 

Vy11 = (eb = €a) a = €pl1 = Call 


Writing the power for the other elements, we have 
Ugt2 = Eci2 — €al2 
313 = €cl3 = €p23 
When we add together the element power terms, we discover that once we collect terms involving a particular 


node voltage, it is multiplied by the sum of currents leaving the node minus the sum of currents entering. 
For example, for node b, we have ey (i3 — i). We see that the currents will obey KCL that multiply each 


27 This content is available online at http: //cnx.org/content/m17317/1.2/. 
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Figure 3.36 


node voltage. Consequently, we conclude that the sum of element powers must equal zero in any 
circuit regardless of the elements used to construct the circuit. 


S > Unik =0 


k 


The simplicity and generality with which we proved this results generalizes to other situations as well. 
In particular, note that the complex amplitudes of voltages and currents obey KVL and KCL, respectively. 
Consequently, we have that 5°, Vid = 0. Furthermore, the complex-conjugate of currents also satisfies 
KCL, which means we also have >, Vid," = 0. And finally, we know that evaluating the real-part of an 
expression is linear. Finding the real-part of this power conservation gives the result that average power 
is also conserved in any circuit. 


1 
yS 5Re Mlk"] = 0 
k 


NOTE: This proof of power conservation can be generalized in another very interesting way. All we 
need is a set of voltages that obey KVL and a set of currents that obey KCL. Thus, for a given 
circuit topology (the specific way elements are interconnected), the voltages and currents can be 
measured at different times and the sum of v-i products is zero. 


S- Uk (ti) Uk (t2) => 0 


k 


Even more interesting is the fact that the elements don’t matter. We can take a circuit and measure 
all the voltages. We can then make element-for-element replacements and, if the topology has not 
changed, we can measure a set of currents. The sum of the product of element voltages and currents 
will also be zero! 


3.17 Electronics” 


So far we have analyzed electrical circuits: The source signal has more power than the output variable, 
be it a voltage or a current. Power has not been explicitly defined, but no matter. Resistors, inductors, 
and capacitors as individual elements certainly provide no power gain, and circuits built of them will not 
magically do so either. Such circuits are termed electrical in distinction to those that do provide power 
gain: electronic circuits. Providing power gain, such as your stereo reading a CD and producing sound, is 
accomplished by semiconductor circuits that contain transistors. The basic idea of the transistor is to let the 
weak input signal modulate a strong current provided by a source of electrical power—the power supply—to 
produce a more powerful signal. A physical analogy is a water faucet: By turning the faucet back and 
forth, the water flow varies accordingly, and has much more power than expended in turning the handle. 
The waterpower results from the static pressure of the water in your plumbing created by the water utility 
pumping the water up to your local water tower. The power supply is like the water tower, and the faucet 


28This content is available online at http: //cnx.org/content/m0035/2.8/. 
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is the transistor, with the turning achieved by the input signal. Just as in this analogy, a power supply is a 
source of constant voltage as the water tower is supposed to provide a constant water pressure. 

A device that is much more convenient for providing gain (and other useful features as well) than the 
transistor is the operational amplifier, also known as the op-amp. An op-amp is an integrated circuit (a 
complicated circuit involving several transistors constructed on a chip) that provides a large voltage gain if 
you attach the power supply. We can model the op-amp with a new circuit element: the dependent source. 


3.18 Dependent Sources” 


A dependent source is either a voltage or current source whose value is proportional to some other voltage 
or current in the circuit. Thus, there are four different kinds of dependent sources; to describe an op-amp, we 
need a voltage-dependent voltage source. However, the standard circuit-theoretical model for a transistor®? 
contains a current-dependent current source. Dependent sources do not serve as inputs to a circuit like 
independent sources. They are used to model active circuits: those containing electronic elements. The 
RLC circuits we have been considering so far are known as passive circuits. 


kv 


Figure 3.37: Of the four possible dependent sources, depicted is a voltage-dependent voltage source in 
the context of a generic circuit. 


Figure 3.38 shows the circuit symbol for the op-amp and its equivalent circuit in terms of a voltage- 
dependent voltage source. 


Rout 
ao—— C 
aO— + 
7 + 
: Rin G(ea-eb) 
bo—- 
bo——_! 
O x O O + O 


Figure 3.38: The op-amp has four terminals to which connections can be made. Inputs attach to nodes 
a and b, and the output is node c. As the circuit model on the right shows, the op-amp serves as an 
amplifier for the difference of the input node voltages. 


Here, the output voltage equals an amplified version of the difference of node voltages appearing across 
its inputs. The dependent source model portrays how the op-amp works quite well. As in most active circuit 
schematics, the power supply is not shown, but must be present for the circuit model to be accurate. Most 
operational amplifiers require both positive and negative supply voltages for proper operation. 


2°This content is available online at http: //cnx.org/content/m0053/2.14/. 
30“Small Signal Model for Bipolar Transistor” http: //cnx.org/content/m1019/latest/ 
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Because dependent sources cannot be described as impedances, and because the dependent variable 
cannot “disappear” when you apply parallel/series combining rules, circuit simplifications such as current 
and voltage divider should not be applied in most cases. Analysis of circuits containing dependent sources 
essentially requires use of formal methods, like the node method (Section 3.15). Using the node method for 
such circuits is not difficult, with node voltages defined across the source treated as if they were known (as 
with independent sources). Consider the circuit shown on the top in Figure 3.39. 


Vout 


Figure 3.39: The top circuit depicts an op-amp in a feedback amplifier configuration. On the bottom 
is the equivalent circuit, and integrates the op-amp circuit model into the circuit. 


Note that the op-amp is placed in the circuit “upside-down,” with its inverting input at the top and 
serving as the only input. As we explore op-amps in more detail in the next section, this configuration will 
appear again and again, and its usefulness demonstrated. To determine how the output voltage is related to 
the input voltage, we apply the node method. Only two node voltages— v and voyt—need be defined; the 
remaining nodes are across sources or serve as the reference. The node equations are 


U — Vin U UV — Vout 


= 24 

R Rin Rr : 3 ) 
Vout — (—Gv) Vout — U Vout 

t t =0 3.25 

Rout Rr Rr 


Note that no special considerations were used in applying the node method to this dependent-source circuit. 
Solving these to learn how vput relates to vin yields 


RrRout 1 1 1 1 1 i 1 1 
t lout = Vin 3.26 
Fa =CRe (Z a zx) (5 7 eT zm) a | meres Be 0 


This expression represents the general input-output relation for this circuit, known as the standard feed- 
back configuration. Once we learn more about op-amps, in particular what its typical element values 
are, the expression will simplify greatly. Do note that the units check, and that the parameter G of the 
dependent source is a dimensionless gain. 
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3.19 Operational Amplifiers” 


Rout 
— c 
a+ 
: Rin G(ea-eb) 
o_o 
bO—— 
O—— O O + O 


Figure 3.40: The op-amp has four terminals to which connections can be made. Inputs attach to nodes 
a and b, and the output is node c. As the circuit model on the right shows, the op-amp serves as an 
amplifier for the difference of the input node voltages. 


Op-amps not only have the circuit model shown in Figure 3.40, but their element values are very special. 


e The input resistance, Ry, is typically large, on the order of 1 MQ. 
e The output resistance, Rout, is small, usually less than 100 2. 
e The voltage gain, G, is large, exceeding 10°. 


The large gain catches the eye; it suggests that an op-amp could turn a 1 mV input signal into a 100 V one. 
If you were to build such a circuit—attaching a voltage source to node a, attaching node b to the reference, 
and looking at the output—you would be disappointed. In dealing with electronic components, you cannot 
forget the unrepresented but needed power supply. 


UNMODELED LIMITATIONS IMPOSED BY POWER SUPPLIES: It is impossible for electronic compo- 
nents to yield voltages that exceed those provided by the power supply or for them to yield currents 
that exceed the power supply’s rating. 


Typical power supply voltages required for op-amp circuits are +15 V. Attaching the 1 mv signal not 
only would fail to produce a 100 V signal, the resulting waveform would be severely distorted. While a 
desirable outcome if you are a rock & roll aficionado, high-quality stereos should not distort signals. Another 
consideration in designing circuits with op-amps is that these element values are typical: Careful control of 
the gain can only be obtained by choosing a circuit so that its element values dictate the resulting gain, 
which must be smaller than that provided by the op-amp. 


31 This content is available online at http: //cnx.org/content/m0036/2.31/. 
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Figure 3.41: The top circuit depicts an op-amp in a feedback amplifier configuration. On the bottom 
is the equivalent circuit, and integrates the op-amp circuit model into the circuit. 


3.19.1 Inverting Amplifier 


The feedback configuration shown in Figure 3.41 is the most common op-amp circuit for obtaining what is 
known as an inverting amplifier. 


RrRout 1 1 1 1 1 1 1 1 
out — DvUin 3.27 
eee (= tate) (Gt +z) ral an ag) 


provides the exact input-output relationship. In choosing element values with respect to op-amp character- 
istics, we can simplify the expression dramatically. 


e Make the load resistance, Ry, much larger than Rout. This situation drops the term 1/Rz, from the 
second factor of (3.27). 
e Make the resistor, R, smaller than Rj,, which means that the 1/ Rj, term in the third factor is negligible. 


With these two design criteria, the expression (3.27) becomes 


Rr 1 1 1 1 
out = Vou 3.28 
te =@Rr (5 i) Z| ae ae ee 
Because the gain is large and the resistance Rout is small, the first term becomes —1/G, leaving us with 
1 1 1 1 1 
. out = in 3.29 
| G ( x) Z| pene a ee 


e If we select the values of Rr and R so that (GR> Rp), this factor will no longer depend on the 


1 
op-amp’s inherent gain, and it will equal “Re 
F 
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Under these conditions, we obtain the classic input-output relationship for the op-amp-based inverting am- 
plifier. 


Vout = ~ R Vin (3.30) 


Consequently, the gain provided by our circuit is entirely determined by our choice of the feedback resistor 
Rr and the input resistor R. It is always negative, and can be less than one or greater than one in 
magnitude. It cannot exceed the op-amp’s inherent gain and should not produce such large outputs that 
distortion results (remember the power supply!). Interestingly, note that this relationship does not depend 
on the load resistance. This effect occurs because we use load resistances large compared to the op-amp’s 
output resistance. Thus observation means that, if careful, we can place op-amp circuits in cascade, without 
incurring the effect of succeeding circuits changing the behavior (transfer function) of previous ones; see this 
problem (Problem 3.43). 


3.19.2 Active Filters 


As long as design requirements are met, the input-output relation for the inverting amplifier also applies 
when the feedback and input circuit elements are impedances (resistors, capacitors, and inductors). 


z “e 
—C) + 
t+ 
re 
| Vout 

Ox 

: . Vout _ _4r 

Figure 3.42: ag 


Example 3.7 
Let’s design an op-amp circuit that functions as a lowpass filter. We want the transfer function 
between the output and input voltage to be 


K 
A(f)= 7) oF 
1+ 4 
where K equals the passband gain and f, is the cutoff frequency. Let’s assume that the inversion 
(negative gain) does not matter. With the transfer function of the above op-amp circuit in mind, 
let’s consider some choices. 
e Zp=K,Z=1+4+ if This choice means the feedback impedance is a resistor and that the 
c 
input impedance is a series combination of an inductor and a resistor. In circuit design, we 


try to avoid inductors because they are physically bulkier than capacitors. 


1 1 
e Zp = —,Z= K Consider the reciprocal of the feedback impedance (its admittance): 


if ’ 
ae 
Zp ot =1t+ iF Since this admittance is a sum of admittances, this expression suggests 


c 


1 
the parallel combination of a resistor (value = 1 () and a capacitor (value = — F). We 


have the right idea, but the values (like 1 Q) are not right. Consider the general RC parallel 
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Figure 3.43 


1 
combination; its admittance is Re +j2a fC. Letting the input resistance equal R, the transfer 
F 


Rp/R 


function of the op-amp inverting amplifier now is H (f) = ~ T+ janfRpC 
JAN J UP 


1 

RrC 

Creating a specific transfer function with op-amps does not have a unique answer. As opposed to design 
with passive circuits, electronics is more flexible (a cascade of circuits can be built so that each has little 
effect on the others; see Problem 3.43) and gain (increase in power and amplitude) can result. To complete 
our example, let’s assume we want a lowpass filter that emulates what the telephone companies do. Signals 
transmitted over the telephone have an upper frequency limit of about 3 kHz. For the second design choice, 
we require ReC = 5.3 x 107°. Thus, many choices for resistance and capacitance values are possible. A 
1 uF capacitor and a 330 2 resistor, 10 nF and 33 kQ, and 10 pF and 33 MQ would all theoretically work. 


Let’s also desire a voltage gain of ten: zl = 10, which means R = Rr Recall that we must have R < Rin. 


R 
Thus, we have the gain equal to - and the cutoff frequency 


As the op-amp’s input impedance is about 1 MQ, we don’t want R too large, and this requirement means 
that the last choice for resistor/capacitor values won’t work. We also need to ask for less gain than the 


op-amp can provide itself. Because the feedback “element” is an impedance (a parallel resistor capacitor 


|ZF| 


combination), we need to examine the gain requirement more carefully. We must have —— < 10° for all 


Rr 


pee) /R < 10°. As this impedance decreases with frequency, the 
F 


frequencies of interest. Thus, ( 


design specification of —" = 10 means that this criterion is easily met. Thus, the first two choices for the 


resistor and capacitor values (as well as many others in this range) will work well. Additional considerations 
like parts cost might enter into the picture. Unless you have a high-power application (this isn’t one) or ask 
for high-precision components, costs don’t depend heavily on component values as long as you stay close to 
standard values. For resistors, having values 10%, easily obtained values of r are 1, 1.4, 3.3, 4.7, and 6.8, 
and the decades span 0-8. 


Exercise 3.17 (Solution on p. 95.) 
What is special about the resistor values; why these rather odd-appearing values for r? 


3.19.3 Intuitive Way of Solving Op-Amp Circuits 


When we meet op-amp design specifications, we can simplify our circuit calculations greatly, so much so that 
we don’t need the op-amp’s circuit model to determine the transfer function. Here is our inverting amplifier. 

When we take advantage of the op-amp’s characteristics—large input impedance, large gain, and small 
output impedance—we note the two following important facts. 


70 CHAPTER 3. ANALOG SIGNAL PROCESSING 


Figure 3.44 


e The current i;, must be very small. The voltage produced by the dependent source is 10° times the 


voltage v. Thus, the voltage v must be small, which means that ij, = must be tiny. For example, 


in 
if the output is about 1 V, the voltage v = 10-°V, making the current ij, = 10~''A. Consequently, 
we can ignore 2j, in our calculations and assume it to be zero. 
e Because of this assumption—essentially no current flow through Rj;,—\the voltage v must also be 
essentially zero. This means that in op-amp circuits, the voltage across the op-amp’s input is basically 
Zero. 


Armed with these approximations, let’s return to our original circuit as shown in Figure 3.44. The node 
voltage e is essentially zero, meaning that it is essentially tied to the reference node. Thus, the current 


through the resistor R equals = Furthermore, the feedback resistor appears in parallel with the load 


resistor. Because the current going into the op-amp is zero, all of the current flowing through R flows 


Vin Rr 


through the feedback resistor (¢¢ = i)! The voltage across the feedback resistor v equals . Because 


the left end of the feedback resistor is essentially attached to the reference node, the voltage across it equals 
VinR 


the negative of that across the output resistor: Vout = —v = a Using this approach makes analyzing 


new op-amp circuits much easier. When using this technique, check to make sure the results you obtain 
are consistent with the assumptions of essentially zero current entering the op-amp and nearly zero voltage 
across the op-amp’s inputs. 


Example 3.8 


Figure 3.45: Two-source, single-output op-amp circuit example. 
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Let’s try this analysis technique on a simple extension of the inverting amplifier configuration shown 
in Figure 3.45. If either of the source-resistor combinations were not present, the inverting amplifier 
remains, and we know that transfer function. By superposition, we know that the input-output 
relation is 


eee 3.31 
U t Ry Vin Ro Vin ( ) 


When we start from scratch, the node joining the three resistors is at the same potential as the 
reference, e ¥ 0, and the sum of currents flowing into that node is zero. Thus, the current 7 flowing 
(1) (2) 

in the resistor Rr equals R + —.. Because the feedback resistor is essentially in parallel with 
the load resistor, the voltages must satisfy v = —vout. In this way, we obtain the input-output 
relation given above. 

What utility does this circuit have? Can the basic notion of the circuit be extended without 
bound? 


3.20 The Diode” 


The resistor, capacitor, and inductor are linear circuit elements in that their v-i relations are linear in the 
mathematical sense. Voltage and current sources are (technically) nonlinear devices: stated simply, doubling 
the current through a voltage source does not double the voltage. A more blatant, and very useful, nonlinear 
circuit element is the diode (learn more®?). Its input-output relation has an exponential form. 


q 
i= Ip (cer? (¢) _ 1) (3.32) 
Here, the quantity q represents the charge of a single electron in coulombs, & is Boltzmann’s constant, and 


A i (HA) 


50 


-1.0 -0.5 | 05 V(V) 


Figure 3.46: v-i relation and schematic symbol for the diode. Here, the diode parameters were room 
temperature and Jp = 1 pA. 


kT 
T is the diode’s temperature in kK. At room temperature, the ratio — = 25 mV. The constant Jp is the 


leakage current, and is usually very small. Viewing this v-i relation in Fiadie 3.46, the nonlinearity becomes 
obvious. When the voltage is positive, current flows easily through the diode. This situation is known as 
forward biasing. When we apply a negative voltage, the current is quite small, and equals Jp, known as 
the leakage or reverse-bias current. A less detailed model for the diode has any positive current flowing 
through the diode when it is forward biased, and no current when negative biased. Note that the diode’s 
schematic symbol looks like an arrowhead; the direction of current flow corresponds to the direction the 
arrowhead points. 


32This content is available online at http: //cnx.org/content/m0037/2.16/. 
33«P_N Junction: Part II” http://cnx.org/content/m1004/latest/ 
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+ 
wi) RSS Vout 


Figure 3.47 


Because of the diode’s nonlinear nature, we cannot use impedances nor series/parallel combination rules 
to analyze circuits containing them. The reliable node method can always be used; it only relies on KVL for 
its application, and KVL is a statement about voltage drops around a closed path regardless of whether 
the elements are linear or not. Thus, for this simple circuit we have 

Mout = Ip. (ext (%—vowt) — 1) (3.33) 

R 

This equation cannot be solved in closed form. We must understand what is going on from basic principles, 
using computational and graphical aids. As an approximation, when vj, is positive, current flows through 
the diode so long as the voltage vout is smaller than v;, (so the diode is forward biased). If the source is 
negative or Vout “tries” to be bigger than v;,, the diode is reverse-biased, and the reverse-bias current flows 
through the diode. Thus, at this level of analysis, positive input voltages result in positive output voltages 
with negative ones resulting in Vpu;, = —RIp. 


A idiode A 


YF Vout 


Figure 3.48 


We need to detail the exponential nonlinearity to determine how the circuit distorts the input voltage 
waveform. We can of course numerically solve the circuit shown in Figure 3.47 to determine the output 
voltage when the input is a sinusoid. To learn more, let’s express this equation graphically. We plot each 
term as a function of Voy; for various values of the input voltage v;,; where they intersect gives us the output 
voltage. The left side, the current through the output resistor, does not vary itself with vin, and thus we 
have a fixed straight line. As for the right side, which expresses the diode’s v-i relation, the point at which 
the curve crosses the Vout axis gives us the value of vin. Clearly, the two curves will always intersect just once 
for any value of vin, and for positive vj, the intersection occurs at a value for Voy, smaller than v;,. This 
reduction is smaller if the straight line has a shallower slope, which corresponds to using a bigger output 
resistor. For negative vin, the diode is reverse-biased and the output voltage equals —RIp. 

What utility might this simple circuit have? The diode’s nonlinearity cannot be escaped here, and the 
clearly evident distortion must have some practical application if the circuit were to be useful. This circuit, 
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known as a half-wave rectifier, is present in virtually every AM radio twice and each serves very different 
functions! We’ll learn what functions later. 


+ 
V; a 
n(_) Vout 


Figure 3.49 


Here is a circuit involving a diode that is actually simpler to analyze than the previous one. We know that 
the current through the resistor must equal that through the diode. Thus, the diode’s current is proportional 
to the input voltage. As the voltage across the diode is related to the logarithm of its current, we see that 


the input-output relation is 
kT in 
ee In ( ee 1) (3.34) 
qd 


Clearly, the name logarithmic amplifier is justified for this circuit. 


3.21 Analog Signal Processing Problems” 
Problem 3.1: Simple Circuit Analysis 


+O >, +O >, +O—>, 
1 1 3 ‘ 
Vv Vv Vv 
2 ail =—=C 
-O 6. -O 
(a) Circuit a (b) Circuit b (c) Circuit c 
Figure 3.50 


For each circuit shown in Figure 3.50, the current i equals cos (2zt). 


(a) What is the voltage across each element and what is the voltage v in each case? 

(b) For the last circuit, are there element values that make the voltage v equal zero for all time? If so, 
what element values work? 

(c) Again, for the last circuit, if zero voltage were possible, what circuit element could substitute for the 
capacitor-inductor series combination that would yield the same voltage? 


Problem 3.2: Solving Simple Circuits 
(a) Write the set of equations that govern Circuit A’s behavior (Figure 3.51). 


34This content is available online at http: //cnx.org/content/m10349/2.42/. 
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(b) Solve these equations for i;: In other words, express this current in terms of element and source values 
by eliminating non-source voltages and currents. 
(c) For Circuit B, find the value for Ry that results in a current of 5 A passing through it. 
d) What is the power dissipated by the load resistor Ry in this case? 
y 


WW Re 
i Ry vin 15 a(A) 202 <RL 
(a) Circuit A (b) Circuit B 


Figure 3.51 


Problem 3.3: Equivalent Resistance 
For each of the circuits shown in Figure 3.52, find the equivalent resistance using series and parallel combi- 
nation rules. 

Calculate the conductance seen at the terminals for circuit (c) in terms of each element’s conductance. 
Compare this equivalent conductance formula with the equivalent resistance formula you found for circuit 
(b). How is the circuit (c) derived from circuit (b)? 


Problem 3.4: Superposition Principle 

One of the most important consequences of circuit laws is the Superposition Principle: The current or 
voltage defined for any element equals the sum of the currents or voltages produced in the element by the 
independent sources. This Principle has important consequences in simplifying the calculation of circuit 
variables in multiple source circuits. 


(a) For the circuit depicted in Figure 3.53, find the indicated current using any technique you like (you 
should use the simplest). 

(b) You should have found that the current i is a linear combination of the two source values: i = 
CyVin + Colin. This result means that we can think of the current as a superposition of two components, 
each of which is due to a source. We can find each component by setting the other sources to zero. 
Thus, to find the voltage source component, you can set the current source to zero (an open circuit) and 
use the usual tricks. To find the current source component, you would set the voltage source to zero 
(a short circuit) and find the resulting current. Calculate the total current i using the Superposition 
Principle. Is applying the Superposition Principle easier than the technique you used in part (1)? 


Problem 3.5: Current and Voltage Divider 


Use current or voltage divider rules to calculate the indicated circuit variables in Figure 3.54. 


Problem 3.6: Thévenin and Mayer-Norton Equivalents 
Find the Thévenin and Mayer-Norton equivalent circuits for the circuits shown in Figure 3.55. 


Problem 3.7: Detective Work 
In the circuit depicted in Figure 3.56, the circuit N, has the v-i relation v; = 32; +7 when 7, = 2. 


(a) Find the Thévenin equivalent circuit for circuit No. 
(b) With i, = 2, determine R such that i; = —1. 


Problem 3.8: Bridge Circuits 
Circuits having the form of one shown in Figure 3.57 are termed bridge circuits. 
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(d) circuit d 
Figure 3.52 
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Figure 3.53 


(a) What resistance does the current source see when nothing is connected to the output terminals? 

(b) What resistor values, if any, will result in a zero voltage for Vout? 

(c) Assume Ry = 10, Ro = 20, R3 = 20 and Ry = 40. Find the current i when the current source ijn is 
Im [(4 + 2j) e/?"?'|. Express your answer as a sinusoid. 


Problem 3.9: Cartesian to Polar Conversion 
Convert the following expressions into polar form. Plot their location in the complex plane*’. 


(a) (1+ v=3)’ (b) 3+ j4 
(c) (2-1) /(2+i—) (d) (4-—39%) (1+ 4/2) 
(e) 3c!” + 4eI7/? (f) (V3 +3) Q/2e—In/4 


8) Ty jan 


Problem 3.10: The Complex Plane 
The complex variable z is related to the real variable u according to 


z=1+e™ 


(a) Sketch the contour of values z takes on in the complex plane. 


35“The Complex Plane” http: //cnx.org/content/m10596/latest/ 
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(c) circuit c 


Figure 3.54 
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(c) circuit c 


Figure 3.55 


(b) What are the maximum and minimum values attainable by |z|? 


(c) Sketch the contour the rational function if 


1 : 
i traces in the complex plane. 


Problem 3.11: Cool Curves 


In the following expressions, the variable x runs from zero to infinity. What geometric shapes do the following 
trace in the complex plane? 
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Figure 3.57 


(a) e/* (b) 1 +e 


(c) e~*ei® (dd) €9® + ef (47/4) 


Problem 3.12: Trigonometric Identities and Complex Exponentials 
Show the following trigonometric identities using complex exponentials. In many cases, they were derived 
using this approach. 


(a) sin2u =2sinucosu (b) cos?u = 4 (1 + cos 2u) 
(c) cos?u + sin?u = 1 (d) #sinu = cosu 
Problem 3.13: Transfer Functions 
Find the transfer function relating the complex amplitudes of the indicated variable and the source shown 


in Figure 3.58. Plot the magnitude and phase of the transfer function over the range of 0.01 Hz to 100 Hz 
on logarithmic vertical and horizontal coordinates. 


KH - 


1 1 
+ ; + . 
Vin C) 1 2-—=v Vin C) 2Sv “in (*) Your “in C) Y ‘out 


(a) circuit a b) circuit b (c) circuit c (d) circuit d 


Figure 3.58 
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Problem 3.14: Using Impedances 
Find the differential equation relating the indicated variable to the source(s) using impedances for each 
circuit shown in Figure 3.59. 


i ee 
lout = C R41 Ro 
Vv + L = lout 
ee = L in A) Vin 
Re 
(a) circuit a (b) circuit b 
pie - >W 
Ly i 1 
in( 4) R Lo C= Vv lin 1 1g 2—— 1< 


(c) circuit c (d) circuit d 


Figure 3.59 


Problem 3.15: Measurement Chaos 
The simple circuit shown in Figure 3.60 was constructed but the signal measurements were made hap- 


2 
hazardly. When the source was sin (27 fot), the current 7 (t) equaled ve sin (27 fot + 7/4) and the voltage 
1 
V(t) = 3 sin (27 fot). 


(a) What is the voltage v (t)? 
(b) Find the impedances Z; and Zp. 
(c) Construct these impedances from elementary circuit elements. 


Problem 3.16: Transfer Functions 
In the circuit shown in Figure 3.61, the voltage source equals vj, (f) = 10 sin (4). 


(a) Find the transfer function between the source and the indicated output voltage. 


. + Vy t — 
I(t). Z 


+ + 
vs) fed vet 


Figure 3.60 
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(b) For the given source, find the output voltage. 


+ 
Vint 2 4 Vout 


Figure 3.61 


Problem 3.17: A Simple Circuit 
You are given this “simple” circuit. 


_ lout 


mls 


Figure 3.62 


(a) What is the transfer function between the source and the indicated output current? 
(b) If the output current is measured to be cos (2t), what was the source? 


Problem 3.18: Circuit Design 


(a) Find the transfer function between the input and the output voltages for the circuits shown in Fig- 
ure 3.63. 

(b) At what frequency does the transfer function have a phase shift of zero? What is the circuit’s gain at 
this frequency? 

(c) Specifications demand that this circuit have an output impedance (its equivalent impedance) less than 
8Q for frequencies above 1 kHz, the frequency at which the transfer function is maximum. Find element 
values that satisfy this criterion. 


“Oot 


Figure 3.63 


Problem 3.19: Equivalent Circuits and Power 
Suppose we have an arbitrary circuit of resistors that we collapse into an equivalent resistor using the series 
and parallel rules. Is the power dissipated by the equivalent resistor equal to the sum of the powers dissipated 
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by the actual resistors comprising the circuit? Let’s start with simple cases and build up to a complete proof. 


(a) Suppose resistors Ry; and Rz are connected in parallel. Show that the power dissipated by (R || R2) 
equals the sum of the powers dissipated by the component resistors. 

(b) Now suppose R; and Ry are connected in series. Show the same result for this combination. 

(c) Use these two results to prove the general result we seek. 


Problem 3.20: Power Transmission 
The network shown in Figure 3.64a represents a simple power transmission system. The generator produces 


| i: IL (os | 
! ort | oH 
+ Rs | i | | 1 
Vv C) | | 100, 100 S—- 
g =" | | | 1 oll 

+ t | — 
ee 

power generator lossy power load 
transmission line 

(a) Simple power transmission system (b) Modified load circuit 


Figure 3.64 


60 Hz and is modeled by a simple Thévenin equivalent. The transmission line consists of a long length of 
copper wire and can be accurately described as a 50 resistor. 


(a) Determine the load current Ry and the average power the generator must produce so that the load 
receives 1,000 watts of average power. Why does the generator need to generate more than 1,000 watts 
of average power to meet this requirement? 

(b) Suppose the load is changed to that shown in Figure 3.64b. Now how much power must the generator 
produce to meet the same power requirement? Why is it more than it had to produce to meet the 
requirement for the resistive load? 

(c) The load can be compensated to have a unity power factor(see Exercise 3.13) so that the voltage 
and current are in phase for maximum power efficiency. —The compensation technique is to place a 
circuit in parallel to the load circuit. What element works and what is its value? 

(d) With this compensated circuit, how much power must the generator produce to deliver 1,000 watts 
average power to the load? 


Problem 3.21: Optimal Power Transmission 

Figure 3.65 shows a general model for power transmission. The power generator is represented by a Thévenin 
equivalent and the load by a simple impedance. In most applications, the source components are fixed while 
there is some latitude in choosing the load. 


(a) Suppose we wanted the maximize “voltage transmission:” make the voltage across the load as large as 
possible. What choice of load impedance creates the largest load voltage? What is the largest load 
voltage? 

(b) If we wanted the maximum current to pass through the load, what would we choose the load impedance 
to be? What is this largest current? 

(c) What choice for the load impedance maximizes the average power dissipated in the load? What is 
most power the generator can deliver? 
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Figure 3.65 


NOTE: One way to maximize a function of a complex variable is to write the expression in terms 
of the variable’s real and imaginary parts, evaluate derivatives with respect to each, set both 
derivatives to zero and solve the two equations simultaneously. 


Problem 3.22: Big is Beautiful 
Sammy wants to choose speakers that produce very loud music. He has an amplifier and notices that the 
speaker terminals are labeled “8 Q source.” 


(a) What does this mean in terms of the amplifier’s equivalent circuit? 

(b) Any speaker Sammy attaches to the terminals can be well-modeled as a resistor. Choosing a speaker 
amounts to choosing the values for the resistor. What choice would maximize the voltage across the 
speakers? 

(c) Sammy decides that maximizing the power delivered to the speaker might be a better choice. What 
values for the speaker resistor should be chosen to maximize the power delivered to the speaker? 


Problem 3.23: Sharing a Channel 

Two transmitter-receiver pairs want to share the same digital communications channel. The transmitter 
signals will be added together by the channel. Receiver design is greatly simplified if first we remove the 
unwanted transmission (as much as possible). Each transmitter signal has the form 


x; (t) = Asin(27f;t) , O<t<T 


where the amplitude is either zero or A and each transmitter uses its own frequency f;. Each frequency is 
harmonically related to the bit interval duration T, where the transmitter one uses the the frequency 1/T. 
The datarate is 10 Mbps. 


(a) Draw a block diagram that expresses this communication scenario. 

(b) Find circuits that the receivers could employ to separate unwanted transmissions. Assume the received 
signal is a voltage and the output is to be a voltage as well. 

(c) Find the second transmitter’s frequency so that the receivers can suppress the unwanted transmission 
by at least a factor of ten. 


Problem 3.24: Circuit Detective Work 
In the lab, the open-circuit voltage measured across an unknown circuit’s terminals equals sin (t). When a 


1 
10 resistor is placed across the terminals, a voltage of a sin (t + 7/4) appears. 
(a) What is the Thévenin equivalent circuit? 
(b) What voltage will appear if we place a 1F capacitor across the terminals? 


Problem 3.25: Mystery Circuit 

We want to determine as much as we can about the circuit lurking in the impenetrable box shown in 
Figure 3.66. A voltage source vin = 2 volts has been attached to the left-hand terminals, leaving the right 
terminals for tests and measurements. 


(a) Sammy measures v = 10 V when a 1 2? resistor is attached to the terminals. Samantha says he is 
wrong. Who is correct and why? 
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+ 
Vin C) Resistors Vv 


Figure 3.66 


(b) When nothing is attached to the right-hand terminals, a voltage of v = 1 V is measured. What circuit 
could produce this output? 

(c) When a current source is attached so that i = 2 A, the voltage v is now 3 V. What resistor circuit 
would be consistent with this and the previous part? 


Problem 3.26: More Circuit Detective Work 
The left terminal pair of a two terminal-pair circuit is attached to a testing circuit as shown in Figure 3.67. 
The test source vin (t) equals sin (¢). 

We make the following measurements. 


1 
e With nothing attached to the terminals on the right, the voltage v (t) equals Va cos (t + 1/4). 


e When a wire is placed across the terminals on the right, the current i (t) was — sin (t). 


Vin ©) Circuit v 


Figure 3.67 


(a) What is the impedance “seen” from the terminals on the right? 
(b) Find the voltage v (t) if a current source is attached to the terminals on the right so that i (t) = sin (¢). 


Problem 3.27: Linear, Time-Invariant Systems 

For a system to be completely characterized by a transfer function, it needs not only be linear, but also 
to be time-invariant. A system is said to be time-invariant if delaying the input delays the output by the 
same amount. Mathematically, if S [x (¢)] = y(t), meaning y(t) is the output of a system S [e] when z (t) 
is the input, S [e] is the time-invariant if S [x(t — r)] = y(t — 7) for all delays 7 and all inputs x(t). Note 
that both linear and nonlinear systems have this property. For example, a system that squares its input is 
time-invariant. 


(a) Show that if a circuit has fixed circuit elements (their values don’t change over time), its input-output 
relationship is time-invariant. Hint: Consider the differential equation that describes a circuit’s input- 
output relationship. What is its general form? Examine the derivative(s) of delayed signals. 

(b) Show that impedances cannot characterize time-varying circuit elements (R, L, and C). Consequently, 
show that linear, time-varying systems do not have a transfer function. 

(c) Determine the linearity and time-invariance of the following. Find the transfer function of the linear, 
time-invariant (LTT) one(s). 
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i) diode ii) y (t) = a (t) sin (27 fot) 
iii) y(t) =a (t—7) iv) y(t) =a(t)+ N(t) 
Problem 3.28: Long and Sleepless Nights 
Sammy went to lab after a long, sleepless night, and constructed the circuit shown in Figure 3.68. He cannot 


remember what the circuit, represented by the impedance Z, was. Clearly, this forgotten circuit is important 
as the output is the current passing through it. 


(a) What is the Thévenin equivalent circuit seen by the impedance? 
(b) In searching his notes, Sammy finds that the circuit is to realize the transfer function 


1 


BO) = FonF +3 


Find the impedance Z as well as values for the other circuit elements. 


- vi 


Vin C) c= |Z 


out 


Figure 3.68 


Problem 3.29: A Testing Circuit 
The simple circuit shown in Figure 3.69 was given on a test. When the voltage source is 5sin(t), the 
current 7 (t) = V2 cos (t — arctan (2) — 1/4). 


(a) What is voltage Uout (t)? 
(b) What is the impedance Z at the frequency of the source? 


i(t 
Mz 


Vin 1 Vout 


Figure 3.69 


Problem 3.30: Black-Box Circuit 

You are given the circuit shown in Figure 3.70 that has two terminals for attaching circuit elements. 
When you attach a voltage source equaling sin (t) to the terminals, the current through the source equals 
Asin (t+ 7/4) — 2sin (4t). When no source is attached (open-circuited terminals), the voltage across the 
terminals has the form Asin (4t + ¢). 


(a) What will the terminal current be when you replace the source by a short circuit? 
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v(t) Circuit 


Figure 3.70 


(b) If you were to build a circuit that was identical (from the viewpoint of the terminals) to the given one, 
what would your circuit be? 
(c) For your circuit, what are A and ¢? 


Problem 3.31: Solving a Mystery Circuit 

Sammy must determine as much as he can about a mystery circuit by attaching elements to the terminal 
and measuring the resulting voltage. When he attaches a 1() resistor to the circuit’s terminals, he measures 
the voltage across the terminals to be 3sin (¢). When he attaches a 1F capacitor across the terminals, the 
voltage is now 3/2 sin (t — 7/4). 


(a) What voltage should he measure when he attaches nothing to the mystery circuit? 


(b) What voltage should Sammy measure if he doubled the size of the capacitor to 2 F and attached it to 
the circuit? 


Problem 3.32: Find the Load Impedance 


The circuit depicted in Figure 3.71 has a transfer function between the output voltage and the source equal 
to 


+ 
Vint 4 Z1) Vout 


Figure 3.71 


(a) Sketch the magnitude and phase of the transfer function. 

(b) At what frequency does the phase equal 7/2? 

(c) Find a circuit that corresponds to this load impedance. Is your answer unique? If so, show it to be so; 
if not, give another example. 


Problem 3.33: Analog “Hum” Rejection 
“Hum?” refers to corruption from wall socket power that frequently sneaks into circuits. “Hum” gets its name 
because it sounds like a persistent humming sound. We want to find a circuit that will remove hum from any 


signal. A Rice engineer suggests using a simple voltage divider circuit consisting of two series impedances 
(Figure 3.72). 


(a) The impedance Z; is a resistor. The Rice engineer must decide between two circuits for the impedance 
Z2 (Figure 3.73). Which of these will work? 
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Figure 3.73 


(b) Picking one circuit that works, choose circuit element values that will remove hum. 
(c) Sketch the magnitude of the resulting frequency response. 


Problem 3.34: An Interesting Circuit 


(a) For the circuit shown in Figure 3.74, find the transfer function. 
(b) What is the output voltage when the input has the form ij, = 5sin (2000zt)? 


6 3 
+ = 
in A) Vout °—] 
2 1 
Figure 3.74 


Problem 3.35: A Simple Circuit 
You are given the circuit depicted in Figure 3.75. 


(a) What is the transfer function between the source and the output voltage? 

(b) What will the voltage be when the source equals sin (t)? 

(c) Many function generators produce a constant offset in addition to a sinusoid. If the source equals 
1+ sin (¢), what is the output voltage? 


Problem 3.36: An Interesting and Useful Circuit 

The circuit depicted in Figure 3.76 has interesting properties, which are exploited in high-performance 
oscilloscopes. The portion of the circuit labeled “Oscilloscope” represents the scope’s input impedance. 
Rg = 1 MQ and C2 = 30 pF (note the label under the channel 1 input in the lab’s oscilloscopes). A probe 
is a device to attach an oscilloscope to a circuit, and it has the indicated circuit inside it. 
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Figure 3.76 


(a) Suppose for a moment that the probe is merely a wire and that the oscilloscope is attached to a circuit 
that has a resistive Thévenin equivalent impedance. What would be the effect of the oscilloscope’s 
input impedance on measured voltages? 

(b) Using the node method, find the transfer function relating the indicated voltage to the source when 
the probe is used. 

(c) Plot the magnitude and phase of this transfer function when R, = 9 MQ and C\ = 2 pF. 

(d) For a particular relationship among the element values, the transfer function is quite simple. Find that 
relationship and describe what is so special about it. 

(e) The arrow through C) indicates that its value can be varied. Select the value for this capacitor to 
make the special relationship valid. What is the impedance seen by the circuit being measured for this 
special value? 


Problem 3.37: A Circuit Problem 
You are given the circuit depicted in Figure 3.77. 


Figure 3.77 
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(a) Find the differential equation relating the output voltage to the source. 
(b) What is the impedance “seen” by the capacitor? 


Problem 3.38: Analog Computers 

Because the differential equations arising in circuits resemble those that describe mechanical motion, we 
can use circuit models to describe mechanical systems. An ELEC 241 student wants to understand the 
suspension system on his car. Without a suspension, the car’s body moves in concert with the bumps in the 
road. A well-designed suspension system will smooth out bumpy roads, reducing the car’s vertical motion. 
If the bumps are very gradual (think of a hill as a large but very gradual bump), the car’s vertical motion 
should follow that of the road. The student wants to find a simple circuit that will model the car’s motion. 
He is trying to decide between two circuit models (Figure 3.78). 


+ + 
Vroad | _ 1== 1>Vcar road ( _ 1 1S Vear 


Figure 3.78 


Here, road and car displacements are represented by the voltages v;oaq (t) and Ucar (t), respectively. 


(a) Which circuit would you pick? Why? 
(b) For the circuit you picked, what will be the amplitude of the car’s motion if the road has a displacement 
given by Uyoaa (t) = 1+ sin (2t)? 


Problem 3.39: Transfer Functions and Circuits 
You are given the network shown in Figure 3.79. 


+ 
Vin( 2> 3/4—— Vout 


Figure 3.79 


(a) Find the transfer function between V;, and Vout. 

(b) Sketch the magnitude and phase of your transfer function. Label important frequency, amplitude and 
phase values. 

(c) Find vout (t) when vin (t) = sin ($ + 7/4). 


Problem 3.40: Fun in the Lab 

You are given an un-openable box that has two terminals sticking out. You assume the box contains a 
circuit. You measure the voltage sin (t + 7/4) across the terminals when nothing is connected to them and 
the current /2cost when you place a wire across the terminals. 


(a) Find a circuit that has these characteristics. 
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(b) You attach a 1 H inductor across the terminals. What voltage do you measure? 


Problem 3.41: Dependent Sources 
Find the voltage voy; in each of the circuits depicted in Figure 3.80. 


Soe 2 
1/3 #3 
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(a) circuit a (b) circuit b 


Figure 3.80 


Problem 3.42: Operational Amplifiers 
Find the transfer function between the source voltage(s) and the indicated output voltage for the circuits 
shown in Figure 3.81. 


Problem 3.43: Op-Amp Circuit 
The circuit shown in Figure 3.82 is claimed to serve a useful purpose. 


(a) What is the transfer function relating the complex amplitude of the output signal, the current Iout, to 
the complex amplitude of the input, the voltage Vi,? 

(b) What equivalent circuit does the load resistor Rz see? 

(c) Find the output current when vi, = Voe~*/7. 


Problem 3.44: Why Op-Amps are Useful 
The cascade of op-amp circuits shown in Figure 3.83 illustrates the reason why op-amp realizations of transfer 
functions are so useful. 


(a) Find the transfer function relating the complex amplitude of the voltage Vout (t) to the source. Show 
that this transfer function equals the product of each stage’s transfer function. 

(b) What is the load impedance appearing across the first op-amp’s output? 

(c) Figure 3.84 illustrates that sometimes “designs” can go wrong. Find the transfer function for this 
op-amp circuit, and then show that it can’t work! Why can’t it? 


Problem 3.45: Operational Amplifiers 
Consider the circuit of Figure 3.85. 


(a) Find the transfer function relating the voltage Vout (t) to the source. 
(b) In particular, Ry = 530 0, Ci = 1 wF, Ro = 5.3 kQ, Co = 0.01 uF, and Rg = Ry = 5.3 kQ. 
Characterize the resulting transfer function and determine what use this circuit might have. 


Problem 3.46: Designing a Bandpass Filter 
We want to design a bandpass filter that has transfer the function 


janf 
CF) (#9 
Here, jf; is the cutoff frequency of the low-frequency edge of the passband and fy, is the cutoff frequency of 
the high-frequency edge. We want f; = 1 kHz and fp, = 10 kHz. 


H(f) =10 
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Figure 3.81 
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Figure 3.83 
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Figure 3.84 


(a) Plot the magnitude and phase of this frequency response. Label important amplitude and phase values 
and the frequencies at which they occur. 


(b) Design a bandpass filter that meets these specifications. Specify component values. 
Problem 3.47: Pre-emphasis or De-emphasis? 


In audio applications, prior to analog-to-digital conversion signals are passed through what is known as a 
pre-emphasis circuit that leaves the low frequencies alone but provides increasing gain at increasingly 
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higher frequencies beyond some frequency fo. De-emphasis circuits do the opposite and are applied after 
digital-to-analog conversion. After pre-emphasis, digitization, conversion back to analog and de-emphasis, 
the signal’s spectrum should be what it was. 

The op-amp circuit in Figure 3.86 has been designed for pre-emphasis or de-emphasis (Samantha can’t 
recall which). 


Re 
R R=1kQ 
Rr =1kQ 
| = C = 80 nF 
Cc — 


Figure 3.86 


(a) Is this a pre-emphasis or de-emphasis circuit? Find the frequency fp that defines the transition from 
low to high frequencies. 

(b) What is the circuit’s output when the input voltage is sin (27 ft), with f = 4kHz? 

(c) What circuit could perform the opposite function to your answer for the first part? 


Problem 3.48: Active Filter 
Find the transfer function of the active filter depicted Figure 3.87. 


Problem 3.49: This is a filter? 
You are given a circuit as shown in Figure 3.88. 


(a) What is this circuit’s transfer function? Plot the magnitude and phase. 
(b) If the input signal is the sinusoid sin (27 fot), what will the output be when fo is larger than the filter’s 
“cutoff frequency?” 


Problem 3.50: Optical Receivers 
In your optical telephone, the receiver circuit had the form shown in Figure 3.89. This circuit served as a 
transducer, converting light energy into a voltage Vout. The photodiode acts as a current source, producing 
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Figure 3.88 


a current proportional to the light intensity falling upon it. As is often the case in this crucial stage, the 


signals are small and noise can be a problem. Thus, the op-amp stage serves to boost the signal and to filter 
out-of-band noise. 


Zf 


NK : —° Vout 


Figure 3.89 


(a) Find the transfer function relating light intensity to vout. 

(b) What should the circuit realizing the feedback impedance Zp be so that the transducer acts as a 5 kHz 
lowpass filter? 

(c) A clever engineer suggests an alternative circuit (Figure 3.90) to accomplish the same task. Determine 
whether the idea works or not. If it does, find the impedance Z;, that accomplishes the lowpass filtering 
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task. If not, show why it does not work. 


NK Fi —-~° Vout 


Figure 3.90 


Problem 3.51: Reverse Engineering 


The circuit depicted in Figure 3.91 has been developed by the TBBG Electronics design group. They are 
trying to keep its use secret; we, representing RU Electronics, have discovered the schematic and want to 
figure out the intended application. Assume the diode is ideal. 


Re 
Ry= 1 kQ 
C =31.8nF || 
Ry Cc 
NY = 
L~| 7 
Vin C) y Vsiii 
@) _ 


Figure 3.91 


(a) Assuming the diode is a short-circuit (it has been removed from the circuit), what is the circuit’s 
transfer function? 

(b) With the diode in place, what is the circuit’s output when the input voltage is sin (27 fot)? 

(c) What function might this circuit have? 
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Solutions to Exercises in Chapter 3 


Solution to Exercise 3.1 (p. 32) 

One kilowatt-hour equals 3,600,000 watt-seconds, which indeed directly corresponds to 3,600,000 joules. 
Solution to Exercise 3.2 (p. 36) 

KCL says that the sum of currents entering or leaving a node must be zero. If we consider two nodes together 
as a “supernode,” KCL applies as well to currents entering the combination. Since no currents enter an entire 
circuit, the sum of currents must be zero. If we had a two-node circuit, the KCL equation of one must 
be the negative of the other, We can combine all but one node in a circuit into a supernode; KCL for the 
supernode must be the negative of the remaining node’s KCL equation. Consequently, specifying n — 1 KCL 
equations always specifies the remaining one. 

Solution to Exercise 3.3 (p. 37) 

The circuit serves as an amplifier having a gain of aoe 
Ri + Re 
Solution to Exercise 3.4 (p. 38) 

The power consumed by the resistor R, can be expressed as 


2 OM 3 
(Ri + Re)? a 


(Vin aay Vout) tout aoa 


Solution to Exercise 3.5 (p. 38) 
1 5 Ry 2 Re 2 
R a R Vin 3 Vin rot 3 Vin 
1 2 (Ri + Ro) (Ri + Ro) 


Solution to Exercise 3.6 (p. 40) 
Replacing the current source by a voltage source does not change the fact that the voltages are identical. 


v 
Consequently, vin = Rotout OF tout = Re This result does not depend on the resistor R,, which means that 
2 


we simply have a resistor (R2) across a voltage source. The two-resistor circuit has no apparent use. 
Solution to Exercise 3.7 (p. 41) 


R R 
Req = TF Po] Rr, Thus, a 10% change means that the ratio ee must be less than 0.1. A 1% change 


means that ~ < 0.01. 


Solution to Bxercias 3.8 (p. 42) 

In a series combination of resistors, the current is the same in each; in a parallel combination, the voltage 
is the same. For a series combination, the equivalent resistance is the sum of the resistances, which will be 
larger than any component resistor’s value; for a parallel combination, the equivalent conductance is the sum 
of the component conductances, which is larger than any component conductance. The equivalent resistance 
is therefore smaller than any component resistance. 

Solution to Exercise 3.9 (p. 44) 


R vj . . . : R 
Vou-= hae and tse = -R, (resistor Rz is shorted out in this case). Thus, veg = RoE and 
R= R, Re 
“tC Ry + Re 


Solution to Exercise 3.10 (p. 46) 

: i 

leq = Rocks and Req => (R3 | Ry + Ro). 

Solution to Exercise 3.11 (p. 51) 

Division by j27f arises from integrating a complex exponential. Consequently, 
1 


V <= | Ve?™F*t dt 
jonf / i 
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Solution to Exercise 3.12 (p. 52) 

For maximum power dissipation, the imaginary part of complex power should be zero. As the complex power 

is given by VI* = |V||Ije?%-®, zero imaginary part occurs when the phases of the voltage and currents 

agree. 

Solution to Exercise 3.13 (p. 52) 

Paye = Vimslrms cos (¢ — 0). The cosine term is known as the power factor. 

Solution to Exercise 3.14 (p. 56) 

The key notion is writing the imaginary part as the difference between a complex exponential and its complex 

conjugate: 

Vel2rft _ Vie s2tft 
2j 

The response to Ve/?"/* is VH (f) e?"/*, which means the response to V*e~J?™F! is V*H (—f) e?7F*. 

As H(-f) = (H (f )*), the Superposition Principle says that the output to the imaginary part is 

Im [VH (f) e??"/"]. The same argument holds for the real part: Re [Ve!?"/"] + Re [VH (f) e??7/"]. 

Solution to Exercise 3.15 (p. 61) 

To find the equivalent resistance, we need to find the current flowing through the voltage source. This current 

equals the current we have just found plus the current flowing through the other vertical 1 Q resistor. This 


Im [Ve??Ft] = 


(3.35) 


6 11 
current equals + = gum making the total current through the voltage source (flowing out of it) 730 


3 
Thus, the equivalent resistance is —2. 
Solution to Exercise 3.16 (p. 62) 
Not necessarily, especially if we desire individual knobs for adjusting the gain and the cutoff frequency. 
Solution to Exercise 3.17 (p. 69) 
The ratio between adjacent values is about VJ/2. 
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Chapter 4 


Frequency Domain 


4.1 Introduction to the Frequency Domain’ 


In developing ways of analyzing linear circuits, we invented the impedance method because it made solving 
circuits easier. Along the way, we developed the notion of a circuit’s frequency response or transfer function. 
This notion, which also applies to all linear, time-invariant systems, describes how the circuit responds to a 
sinusoidal input when we express it in terms of a complex exponential. We also learned the Superposition 
Principle for linear systems: The system’s output to an input consisting of a sum of two signals is the sum 
of the system’s outputs to each individual component. 

The study of the frequency domain combines these two notions—a system’s sinusoidal response is easy to 
find and a linear system’s output to a sum of inputs is the sum of the individual outputs—to develop the 
crucial idea of a signal’s spectrum. We begin by finding that those signals that can be represented as a 
sum of sinusoids is very large. In fact, all signals can be expressed as a superposition of sinusoids. 

As this story unfolds, we’ll see that information systems rely heavily on spectral ideas. For example, 
radio, television, and cellular telephones transmit over different portions of the spectrum. In fact, spectrum 
is so important that communications systems are regulated as to which portions of the spectrum they can use 
by the Federal Communications Commission in the United States and by International Treaty for the world 
(see Frequency Allocation Chart in Section 7.3). Calculating the spectrum is easy: The Fourier transform 
defines how we can find a signal’s spectrum. 


4.2 Fourier Series’ 


In an earlier module (Exercise 2.4), we showed that a square wave could be expressed as a superposition of 
pulses. As useful as this decomposition was in this example, it does not generalize well to other periodic 
signals: How can a superposition of pulses equal a smooth signal like a sinusoid? Because of the importance 
of sinusoids to linear systems, you might wonder whether they could be added together to represent a large 
number of periodic signals. You would be right and in good company as well. Leonhard Euler? and Carl 
Friedrich Gauss* in particular worried about this problem, and Jean Baptiste Fourier® got the credit even 
though tough mathematical issues were not settled until later. They worked on what is now known as the 
Fourier series: representing any periodic signal as a superposition of sinusoids. 

But the Fourier series goes well beyond being another signal decomposition method. Rather, the Fourier 
series begins our journey to appreciate how a signal can be described in either the time-domain or the 
frequency-domain with no compromise. Let s(t) be a periodic signal with period T. We want to show 
that periodic signals, even those that have constant-valued segments like a square wave, can be expressed 


1This content is available online at http: //cnx.org/content/m0038/2.10/. 
?This content is available online at http: //cnx.org/content/m0042/2.28/. 
Shttp://www-groups.dcs.st-and.ac.uk/~history/Biographies/Euler .html 
4nttp: //www-groups.dcs.st-and.ac.uk/~history/Biographies/Gauss .html 
Shttp://www-groups.dcs.st-and.ac.uk/~history/Biographies/Fourier.html 
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as sum of harmonically related sine waves: sinusoids having frequencies that are integer multiples of 
the fundamental frequency. Because the signal has period T, the fundamental frequency is 1/T. The 
Fourier series expresses the signal as a superposition of complex exponentials having frequencies k/T, k = 


{62 B10; Tyce} 
ve .2akt 
sa \) see F: (4.1) 
k=—0o 
The zeroth coefficient equals the signal’s average value and is real-valued for real-valued signals: co = 
i 
a fos(t) dt. In general, the other Fourier coefficients are complex-valued; we will find many special cases 
. @akt 

wherein come coefficients are real-valued. The family of functions {é ‘a, are called basis functions and 


form the foundation of the Fourier series. No matter what the periodic signal might be, these functions are 
always present and form the representation’s building blocks. They depend on the signal period T, and are 
indexed by k. 


KEY POINT: Assuming we know the period, knowing the Fourier coefficients is equivalent to 
knowing the signal. Thus, it makes no difference if we have a time-domain or a frequency-domain 
characterization of the signal. 


Exercise 4.1 (Solution on p. 139.) 
What is a sinusoid’s Fourier series? You can find this expression by inspection! 


To find the Fourier coefficients, we note the orthogonality property 


To onkt — .2rlt T. ifk=I 
i fret tT dai | = Ti(k—1) (4.2) 
0 0, ifkél 


5(n) is the unit-sample defined in equation (2.28). Using this definition, 6(& — 1) equals one when k —1 =0 
(k = 1) and zero when k —-1 40 (k £1). Assuming for the moment that the Fourier series “works,” we can 
find a signal’s Fourier coefficients, its spectrum, by exploiting this orthogonality property. Simply multiply 
each side of (4.1) by e~/?*"" and integrate over the interval [0,7]. 


Ae ee 2nlt 
C= z/ s(t)e 7 T dt (4.3) 
L Jo 


Example 4.1 
Finding the Fourier series coefficients for the square wave sq,y (t) is very simple. Mathematically, 
this signal can be expressed over one period as 


1, 0<t<F 
sqr (t) = : 7 
ar () ei LeGay 
The expression for the Fourier coefficients has the form 


1 fz jem 1 ie jon i 
= “4 T dt- = “7 TT dt : 
Ck a, e fee (4.4) 


Lr 
2 


NOTE: When integrating an expression containing 7, treat it just like any other constant. 


The two integrals are very similar, one equaling the negative of the other. The final expression 
becomes 


ae ack (( 1)" 1) 
ti k odd (4.5) 


0 k even 
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Thus, the Fourier series for the square wave is 


sq (t) = St gee (4.6) 


KE f-45—3,—151,3,.83} jak 


Consequently, the square wave equals a sum of complex exponentials, but only those having fre- 
quencies equal to odd multiples of the fundamental frequency r The coefficients decay slowly as 
the frequency index k increases. This index corresponds to the k-th harmonic of the signal’s period. 


4.2.1 Fourier coefficients properties 
A signal’s Fourier series spectrum c, has interesting properties. 

Property 4.1: 

If s(t) is real, c, = c*, (real-valued periodic signals have conjugate-symmetric spectra). 
This result follows from the integral that calculates the cy, from the signal. Furthermore, this result means 
that Re[c,] = Re[c_x]: The real part of the Fourier coefficients for real-valued signals is even. Similarly, 
Im [cx] = —Im [c_,]: The imaginary parts of the Fourier coefficients have odd symmetry. Consequently, if 
you are given the Fourier coefficients for positive indices and zero and are told the signal is real-valued, you 
can find the negative-indexed coefficients, hence the entire spectrum. This kind of symmetry, cy, = c*,, is 
known as conjugate symmetry. 

Property 4.2: 

If s(—t) = s(t), which says the signal has even symmetry about the origin, c_, = cx. 
Given the previous property for real-valued signals, the Fourier coefficients of even signals are real-valued. 
A real-valued Fourier expansion amounts to an expansion in terms of only cosines, which is the simplest 
example of an even signal. 

Property 4.3: 

If s(—t) = —s(t), which says the signal has odd symmetry, c_, = —Cp. 
Therefore, the Fourier coefficients are purely imaginary. The square wave is a great example of an odd- 
symmetric signal. 


Property 4.4: 
janrkr 


The spectral coefficients for a periodic signal delayed by 7, s(¢— 7), arec,ze. T  , where cy denotes 
the spectrum of s(t). Delaying a signal by 7 seconds results in a spectrum having a linear phase 


shiteorc oe” 


magnitude is unaffected. Showing this property is easy. 


aa a . amkt tf? * 2mk(t+r) 
z/ s(t—T)e? T a=z | s(the ? T dt 
0 —T 
4. 
1 .2akt Tor .2akt ( 0 
= pe T i Ss (t) e? T dt 


—T 


in comparison to the spectrum of the un-delayed signal. Note that the spectral 


Note that the range of integration extends over a period of the integrand. Consequently, it should 
not matter how we integrate over a period, which means that aa (-)dt = fee (-) dt, and we have 
our result. 


By the way, you can easily show that no matter how you integrate a periodic signal over a period, you get 
the same answer. Let x(t) be period with period T. 


[roa froas [moa 
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Work on the first integral 


-[ tu T)du+ f° “a(t tu =t+T] 


T-T 0) 


T T-T 
us / serait i, a(t)dt [e(u—T) = a(t)] 


T-T 


Reversing the order of the intergrals and combining them, we get 


T-T T 
/ x(t) dt = ih x(t) dt 
—T 0 
Property 4.5: 


The Fourier series obeys Parseval’s Theorem,” one of the most important results in signal anal- 
ysis. Parseval’s Theorem states that you can calculate a signal’s power in either the time domain 
or the frequency domain with very similar formulas. 


6 


Theorem 4.1: Parseval’s Theorem 


Average power calculated in the time domain equals the power calculated in the frequency domain. 
lo) 


iT 
= | s(t)dt= S> lex? (4.8) 


k=—oo 


Proof: 

Proving Parseval’s Theorem is easy when we use the orthogonality property of harmonically related 
complex exponentials (equation 4.2). First of all, re-write the square of the signal as a magnitude- 
squared and substitute the Fourier series of the signal into the integral. 


T Ts oo Ge On 
L ffeqa-2 f (> ae) (x oot) at 
0 0 k=—66 = 
1 2 T omkt .2nlt 
Gio et cxet | eo Rue da 
0 


k=—co l=—00 


T onkt .2Qnlt 
Using the fact that if e’ T e T dt=T6(k +1), we find that 


0 
ta = 
z/ s(t)? dt= So cee 
0 k=—0oo 
- S- |e, |? [For real x(t),c_, = ci] 
k=—0o 


4.2.2 A signal’s spectrum 


The Fourier series expression of (4.2) has a very deep interpretation. The time-domain signal s(t) is there 
expressed as a weighted sum of complex exponentials that differ only in their frequency k/T. In this way, 
we have a frequency-domain representation of the signal. The “deep” part is that either representation can 
be used to characterize a signal: we can specify a signal in the time-domain or in the frequency domain. 
Because of the Fourier series, we can find how to represent the signal in the “other” domain. How the Fourier 


6Parseval’s biography can be found at http://www-history.mcs.st-andrews.ac.uk/Biographies/Parseval .html. 
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coefficients depend on frequency is termed the signal’s spectrum. When we plot the magnitude and phase 
of the coefficients to display which harmonics are present in a periodic signal, we are plotting the spectrum. 
Let’s calculate the spectrum of the periodic pulse signal shown in Figure 4.1. 


A pit) 
A | | 
cas KAS 
i} i} i} i 
T 


Figure 4.1: Periodic pulse signal. 


The pulse width is A, the period T’,, and the amplitude A. The spectrum of this signal is given by 


1 fd r j2mkt ; A j2nkA ; 
ame [ 7 j2nk (« ) 


At this point, simplifying this expression requires knowing an interesting property. 


1—¢ 8 =e 98/2. Ce ao 7 10/2) = ¢ 99/2. 27 sin (5) 


Armed with this result, we can simply express the Fourier series coefficients for our pulse sequence. 


as : ThA 
ee a sin ("7") (4.9) 
Tk 
Because this signal is real-valued, we find that the coefficients do indeed have conjugate symmetry: cz = c_x”*. 
Because of this property, we do not need to plot the spectrum for negative k; it can easily be found from the 
spectrum for positive k. The magnitude of a real-valued signal’s spectrum has even symmetry, 
the angle has odd symmetry. 
\c_p| = \cx| ZC_k = —ZCK 


The periodic pulse signal has neither even nor odd symmetry in the time domain; consequently, neither 
properties 4.2 or 4.3 apply. To plot the spectrum, we need to calculate its magnitude and phase. 


wa) (ThA 
|cx| = A ew) (4.10) 
wk 
in (ZEA 
Lc, = — ane + meg pn a) sign (k) 
T wk; 


The function neg (-) equals —1 if its argument is negative and zero otherwise. The somewhat complicated 
expression for the phase results because the sine term in (4.9) can be negative. Because magnitudes must 
be positive, the occasional negative values of the sine function must be accounted for by a phase shift of 7. 
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Figure 4.2: The magnitude and phase of the periodic pulse sequence’s spectrum is shown for positive- 
frequency indices. Here 2 = 0.2 and A= 1. 


Also note the presence of a linear phase term (the first term in Zc, is proportional to frequency k/T). 
Comparing this term with that predicted from delaying a signal, a delay of A/2 is present in our signal. 
Advancing the signal by this amount centers the pulse about the origin, leaving an even signal, which in 
turn means that its spectrum is real-valued. Thus, our calculated spectrum is consistent with the properties 
of the Fourier spectrum. 


Exercise 4.2 (Solution on p. 139.) 
What is the value of co? Recalling that this spectral coefficient corresponds to the signal’s average 
value, does your answer make sense? 


The phase plot shown in Figure 4.2 (Periodic Pulse Sequence) requires some explanation as it does not seem 
to agree with what (4.10) suggests. There, the phase has a linear component, with a jump of 7 every time 
the sinusoidal term changes sign. We must realize that any integer multiple of 27 can be added to a phase 
at each frequency without affecting the value of the complex spectrum. We see that at frequency index 4, 
the phase is nearly —7. The phase at index 5 is undefined because the magnitude is zero in this example. At 
index 6, the formula suggests that the phase of the linear term should be less than (more negative than) —7. 
In addition, we expect a shift of —a in the phase between indices 4 and 6. Thus, the phase value predicted 
by the formula is a little less than —27. Because we can add 27 without affecting the value of the spectrum 
at index 6, the result is a slightly negative number as shown. Thus, the formula and the plot do agree. In 
phase calculations like those made in MATLAB, values are usually confined to the range [—7, 7) by adding 
some (possibly negative) multiple of 27 to each phase value. 

Another aspect of the spectrum needs to be emphasized and this example demonstrates it well. Note 
that the period T enters into the spectrum only through the ratio of the pulse width A and the period. If 
the period is changed in such a way that this ratio remains constant, the spectral plot in Figure (4.2) is 
unchanged when considered as a function of coefficient index k. However, when we plot a spectrum as a 
function of frequency k/T, the plot stretches or compresses horizontally as we change the period. We will 
find that what is important in applications is what frequencies, not what harmonic indices, are contained in 
a periodic signal. 
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4.3 Classic Fourier Series’ 


The classic Fourier series as derived originally by Euler and Fourier expressed a periodic signal (period T) 
in terms of harmonically related sines and cosines. 


= Qrkt ais ; Qrkt 
s(t) = ort De antes (=) ee (=) (4.11) 


The Fourier series and the sine-cosine series are different versions of the same idea, each 
representing a signal’s spectrum. The classic Fourier coefficients, a, and by, express the real and imaginary 
parts respectively of the spectrum while the coefficients c, of the Fourier series express the spectrum as a 
magnitude and phase. Equating the classic Fourier series (4.11) to the Fourier series (4.1), an extra factor 
of two and a complex conjugate become necessary to relate the Fourier coefficients in each. 


Exercise 4.3 (Solution on p. 139.) 
Derive this relationship between the coefficients of the two Fourier series. 


Just as with the Fourier series, we can find the sine-cosine Fourier coefficients using the orthogonality 
properties of sinusoids. Note that the cosine and sine of harmonically related frequencies, even the same 
frequency, are orthogonal. 


T 
2rkt QrIt 
in | —— — )dt=0, kEZIlEZ 
ff sin (AF) 00s (= . EZ LE 


T ae 
2akt 2alt — = 
‘ sin (73 ) sin (25) ae = me ifk=landk #0 and!40 
0 T L 0, k#lork=0=1 (4.12) 
T 


, k=landk#0and140 


[« _ ( nkt anlt\ »,_ 2 
: OS T cos T = F; k=0=l1 


0, k#Al 


These orthogonality relations follow from the following important trigonometric identities. 


sin (a) sin (8) = 5 (cos (a — B) — cos(a + B)) 
cos (a) cos (8) = 5 (cos (a + B) + cos (a — B)) (4.13) 
sin (a) cos (3) = 5 (sin (a + 8) +sin (a — )) 


These identities allow you to substitute a sum of sines and/or cosines for a product of them. Each term in 
the sum can be integrated by noticing one of two important properties of sinusoids. 


e The integral of a sinusoid over an integer number of periods equals zero. 
e The integral of the square of a unit-amplitude sinusoid over a period T equals T/2. 


To use these, let’s, for example, multiply the Fourier series for a signal by the cosine of the /** harmonic 


2alt 
cos (=F) and integrate. The idea is that, because integration is linear, the integration will sift out all but 


‘This content is available online at http: //cnx.org/content/m0039/2.22/. 
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the term involving ay. 


T T co T 
QrIt QrIt Qrkt QrIt 
| Ss (t) COs (F) dt = | ag COS (F) dt + a | COs (=) Cos (#) dt 
k=1 
oo T 
2Qrkt 2QrIt 
+ S- be f sin (=) cos (F) dt 


k=1 


(4.14) 


The first and third terms are zero; in the second, the only non-zero term in the sum results when the indices 


T 
k and | are equal (but not zero), in which case we obtain oe If k = 0 = 1, we obtain agT. Consequently, 


a2 [st Joos (=) at, 10 


All of the Fourier coefficients can be found similarly. 


1 T 
== t) dt 
a= : s(t) 
2/7 Qnkt 
a=7 f s(t) cos ( =") dt , k#0 (4.15) 
2 fF Qkt 
Exercise 4.4 (Solution on p. 139.) 
The expression for ag is referred to as the average value of s(t). Why? 
Exercise 4.5 (Solution on p. 139.) 


What is the Fourier series for a unit-amplitude square wave? 


Example 4.2 
Let’s find the Fourier series representation for the half-wave rectified sinusoid shown in Figure 4.5 


(page 108). , rn 
. Tt 
s(t) = sin( T aa (4.16) 


We have a choice: we can find the complex spectral coefficients c, or we can find the classic 
series coefficients. Given one result, we can find the other. In general, finding the complex-valued 
coefficients is easier, but in this case let’s find the classic series coefficients because of the presence 
of the sinusoid in the signal’s expression. Begin with the sine terms in the series; to find b, we must 


calculate the integral 
a 
2 [? Qrt Qakt 
by = z | sin (=) sin (=) dt (4.17) 


Using our trigonometric identities turns our integral of a product of sinusoids into a sum of integrals 
of individual sinusoids, which are much easier to evaluate. 


[sn (224) sn (728) ae = Lf foo (7 une) cos (EDEN) a 
-{3 a 
0 


(4.18) 


otherwise 
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Thus, 


b= = bo =bg=---=0 


On to the cosine terms. The average value, which corresponds to ag, equals 1/7. The remainder 
of the cosine coefficients are easy to find, but yield the complicated result 


2 1 
—-—-——__k 2,4, 06. 
ap = a k2 —1 € {2,4, } 


0 k odd 


(4.19) 


Thus, the Fourier series for the half-wave rectified sinusoid has non-zero terms for its average, 
fundamental, and even harmonics. The expression for the complex series coefficients concisely 
represents this structure. Because the signal is real-valued, conjugate symmetry applies to the 
coefficients; we need only specify c, for k > 0. 


1 
— k=0 
41 
ian ies 
—-——_ k=2,4 
nk?—1 oot 
0 a8, Gace 


4.4 A Signal’s Power Spectrum* 


A periodic signal, such as the half-wave rectified sinusoid, consists of a sum of elemental sinusoids. A plot 
of the Fourier coefficients as a function of the frequency index, such as shown in Figure 4.3 (Fourier Series 
spectrum of a half-wave rectified sine wave), displays the signal’s spectrum. The word “spectrum” implies 
that the independent variable, here k, corresponds somehow to frequency. Each coefficient is directly related 
to a sinusoid having a frequency of k/T. Thus, if we half-wave rectified a 1 kHz sinusoid, k = 1 corresponds 
to 1 kHz, k = 2 to 2 kHz, etc. 
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Figure 4.3: The Fourier series spectrum of a half-wave rectified sinusoid is shown. The index indicates 
the multiple of the fundamental frequency at which the signal has energy. The left column shows the 
spectrum in terms of the classic Fourier series, the right in terms of the Fourier series coefficients. We 
will find the latter representation more useful. 


8This content is available online at http: //cnx.org/content/m0040/2.20/. 
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Figure 4.4: Power spectrum of a half-wave rectified sinusoid. 


A subtle, but very important, aspect of the Fourier spectrum is its uniqueness: You can unambiguously 
find the spectrum from the signal (decomposition (4.15)) and the signal from the spectrum (composition). 
Thus, any aspect of the signal can be found from the spectrum and vice versa. A signal’s frequency 
domain expression is its spectrum. A periodic signal can be defined either in the time domain (as a 
function) or in the frequency domain (as a spectrum). 

A fundamental aspect of solving electrical engineering problems is whether the time or frequency domain 
provides the most understanding of a signal’s properties and the simplest way of manipulating it. The 
uniqueness property says that either domain can provide the right answer. As a simple example, suppose 
we want to know the (periodic) signal’s maximum value. Clearly the time domain provides the answer 
directly. To use a frequency domain approach would require us to find the spectrum, form the signal from 
the spectrum and calculate the maximum; we’re back in the time domain! 

Another feature of a signal is its average power. A signal’s instantaneous power is defined to be its 
square. The average power is the average of the instantaneous power over some time interval. For a periodic 
signal, the natural time interval is clearly its period; for non-periodic signals, a better choice would be entire 
time or time from onset. For a periodic signal, the average power is the square of its root-mean-squared 
(rms) value. We define the rms value of a periodic signal to be 


| T 
rms (s) = a 8s? (t) dt (4.20) 


r 
power (s) = rms? (s) = a s” (t) dt (4.21) 


Exercise 4.6 (Solution on p. 139.) 
What is the rms value of the half-wave rectified sinusoid? 


and thus its average power is 


To find the average power in the frequency domain, we only need Parseval’s Theorem. 


1 ae ~ 2 2 if ~ 2 2 
z/f 8 (de= De en =a + 5D, (ak + Hh) (4.22) 


It could well be that computing the sum is easier than integrating the signal’s square. Furthermore, 
the contribution of each term in the Fourier series toward representing the signal can be measured by its 
contribution to the signal’s average power. Thus, the power contained in a signal at its kth harmonic is 

245 2 
oe or [cx |”. The power spectrum, P, (k), such as shown in Figure 4.4, plots each harmonic’s 


contribution to the total power. 
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Exercise 4.7 (Solution on p. 139.) 
In high-end audio, deviation of a sine wave from the ideal is measured by the total harmonic 
distortion, which equals the total power in the harmonics higher than the first compared to the 
signal’s total power. Find an expression for the total harmonic distortion for any periodic signal. 
Is this calculation most easily performed in the time or frequency domain? 


4.5 Fourier Series Approximation of Signals’ 


It is interesting to consider the sequence of signals that we obtain as we incorporate more terms into the 
Fourier series approximation of the half-wave rectified sine wave (Example 4.2). Define sx (t) to be the 
signal containing only those terms up to and including the K* harmonic. 


K K 
Qrkt Qrkt 
SK (t) y Ap COS (=) + ) by sin (=) 


(4.23) 


Figure 4.5 shows how this sequence of signals portrays the signal more accurately as more terms are added. 
We need to assess quantitatively the accuracy of the Fourier series approximation so that we can judge 

how rapidly the series approaches the signal. When we use a truncated series, the error—the difference 

between the signal and the K‘t>-harmonic series—corresponds to the unused terms from the series. 


= 2rkt <= ; Qrkt 
€x (t) = fat cos (=) + Pry by sin (=) 
ae = (4.24) 
kt 
|k|=K+1 


To find the rms error, we must square this expression and integrate it over a period. Again, the integral of 


most cross-terms is zero, leaving 
Co 


rms” (ex) = ; S- (az, + dj) 
k=K41 


= np 


k=K+1 


(4.25) 


Figure 4.6 shows how the error in the Fourier series for the half-wave rectified sinusoid decreases as more 
terms are incorporated. In particular, the use of four terms, as shown in the bottom plot of Figure 4.5, has 
arms error (relative to the rms value of the signal) of about 3%. The Fourier series in this case converges 
quickly to the signal. 

We can look at Figure 4.7 to see the power spectrum and the rms approximation error for the square 
wave. Because the Fourier coefficients decay more slowly here than for the half-wave rectified sinusoid, the 
rms error is not decreasing quickly. Said another way, the square-wave’s spectrum contains more power at 
higher frequencies than does the half-wave-rectified sinusoid. This difference between the two Fourier series 


1 
results because the half-wave rectified sinusoid’s Fourier coefficients are proportional to 72 while those of the 


1 
square wave are proportional to —. If fact, after 99 terms of the square wave’s approximation, the error is 


bigger than 10 terms of the approximation for the half-wave rectified sinusoid. Mathematicians have shown 
that no signal has an rms approximation error that decays more slowly than it does for the square wave. 


°This content is available online at http: //cnx.org/content/m10687/2.9/. 
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Fourier Series spectrum of a half-wave rectified sine wave 
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Figure 4.5: The Fourier series spectrum of a half-wave rectified sinusoid is shown in the upper portion. 
The index indicates the multiple of the fundamental frequency at which the signal has energy. The 
cumulative effect of adding terms to the Fourier series for the half-wave rectified sine wave is shown in 
the bottom portion. The dashed line is the actual signal, with the solid line showing the finite series 
approximation to the indicated number of terms, K + 1. 


Exercise 4.8 (Solution on p. 140.) 
Calculate the harmonic distortion for the square wave. 


More than just decaying slowly, Fourier series approximation shown in Figure 4.8 exhibits interesting behav- 
ior. Although the square wave’s Fourier series requires more terms for a given representation accuracy, when 
comparing plots it is not clear that the two are equal. Does the Fourier series really equal the square wave 
at all values of t? In particular, at each step-change in the square wave, the Fourier series exhibits a peak 
followed by rapid oscillations. As more terms are added to the series, the oscillations seem to become more 
rapid and smaller, but the peaks are not decreasing. For the Fourier series approximation for the half-wave 
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Relative rms error 


Figure 4.6: The rms error calculated according to (4.25) is shown as a function of the number of terms 
in the series for the half-wave rectified sinusoid. The error has been normalized by the rms value of the 
signal. 
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Figure 4.7: The upper plot shows the power spectrum of the square wave, and the lower plot the rms 
error of the finite-length Fourier series approximation to the square wave. The asterisk denotes the rms 
error when the number of terms K in the Fourier series equals 99. 


rectified sinusoid (Figure 4.5), no such behavior occurs. What is happening? 

Consider this mathematical question intuitively: Can a discontinuous function, like the square wave, be 
expressed as a sum, even an infinite one, of continuous signals? One should at least be suspicious, and in fact, 
it can’t be thus expressed. The extraneous peaks in the square wave’s Fourier series never disappear; they 
are termed Gibbs’ phenomenon after the American physicist Josiah Willard Gibbs. They occur whenever 
the signal is discontinuous, and will always be present whenever the signal has jumps. This issue brought 
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K=49 


Figure 4.8: Fourier series approximation to sq (t). The number of terms in the Fourier sum is indicated 
in each plot, and the square wave is shown as a dashed line over two periods. 


Fourier!® much criticism from the French Academy of Science (Laplace, Lagrange, Monge and LaCroix 


comprised the review committee) for several years after its presentation on 1807. It was not resolved for 
almost a century, with ultimate solution interesting and important to understand. 

Let’s return to the question of equality; how can the equal sign in the definition of the Fourier series be 
justified? The partial answer is that point wise—each and every value of t—equality is not guaranteed. 
However, mathematicians later in the nineteenth century showed that the rms error of the Fourier series was 
always zero. 

lim rms (ex) = 0 
K-0o 


What this means is that the error between a signal and its Fourier series approximation may not be zero, 
but that its rms value will be zero! It is through the eyes of the rms value that we redefine equality: The 
usual definition of equality is called point wise equality: Two signals sj (t), s2(t) are said to be equal 
point wise if s; (¢) = s(t) for all values of t. A new definition of equality is mean-square equality: Two 
signals are said to be equal in the mean square if rms (s; — s2) = 0. For Fourier series, Gibbs’ phenomenon 
peaks have finite height and zero width. The error differs from zero only at isolated points—whenever the 
periodic signal contains discontinuities—and equals about 9% of the size of the discontinuity. The value of 
a function at a finite set of points does not affect its integral. This effect underlies the reason why defining 
the value of a discontinuous function at its discontinuity, like when we defined the step function on page 17, 


l0nttp: //www-groups.dcs.st-and.ac.uk/~history/Biographies/Fourier .html 
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is has no effect: Whatever you pick for a value has no practical relevance for either the signal’s spectrum or 
for how a system responds to the signal. The Fourier series value “at” the discontinuity is the average of the 
values on either side of the jump. 


4.6 Encoding Information in the Frequency Domain” 


To emphasize the fact that every periodic signal has both a time and frequency domain representation, we 
can exploit both to encode information into a signal. Refer to the Fundamental Model of Communication 
(Figure 1.3). We have an information source, and want to construct a transmitter that produces a signal 
x(t). For the source, let’s assume we have information to encode every T seconds. For example, we want 
to represent typed letters produced by an extremely good typist (a key is struck every T seconds). Let’s 
consider the Fourier series formula in the light of trying to encode information. 


.2akt 


K 
x(t) = y che. (4.26) 
k=—K 


We use a finite sum here merely for simplicity (fewer parameters to determine). An important aspect of 
the spectrum is that each frequency component cz can be manipulated separately: Instead of finding the 
Fourier spectrum from a time-domain specification, let’s construct it in the frequency domain by selecting 
the c, according to some rule that relates coefficient values to the alphabet. In defining this rule, we want to 
always create a real-valued signal x (¢t). Because of the Fourier spectrum’s properties (Property 4.1, p. 99), 
the spectrum must have conjugate symmetry. This requirement means that we can only assign positive- 
indexed coefficients (positive frequencies), with negative-indexed ones equaling the complex conjugate of the 
corresponding positive-indexed ones. 

Assume we have N letters to encode: {a1,...,an}. One simple encoding rule could be to make a single 
Fourier coefficient be non-zero and all others zero for each letter. For example, if a, occurs, we make c, = 1 
and cy = 0, k #4 n. In this way, the n** harmonic of the frequency 1/T is used to represent a letter. Note 
that the bandwidth—the range of frequencies required for the encoding—equals N/T. Another possibility 
is to consider the binary representation of the letter’s index. For example, if the letter a,3 occurs, converting 
13 to its base-2 representation, we have 13 = 1101). We can use the pattern of zeros and ones to represent 
directly which Fourier coefficients we “turn on” (set equal to one) and which we “turn off.” 


Exercise 4.9 (Solution on p. 140.) 
Compare the bandwidth required for the direct encoding scheme (one nonzero Fourier coefficient 
for each letter) to the binary number scheme. Compare the bandwidths for a 128-letter alphabet. 
Since both schemes represent information without loss — we can determine the typed letter uniquely 
from the signal’s spectrum — both are viable. Which makes more efficient use of bandwidth and 
thus might be preferred? 


Exercise 4.10 (Solution on p. 140.) 
Can you think of an information-encoding scheme that makes even more efficient use of the spec- 
trum? In particular, can we use only one Fourier coefficient to represent N letters uniquely? 


We can create an encoding scheme in the frequency domain (p. 111) to represent an alphabet of letters. But, 
as this information-encoding scheme stands, we can represent one letter for all time. However, we note that 
the Fourier coefficients depend only on the signal’s characteristics over a single period. We could change 
the signal’s spectrum every T as each letter is typed. In this way, we turn spectral coefficients on and off as 
letters are typed, thereby encoding the entire typed document. For the receiver in the Fundamental Model 
of Communication (Figure 1.3) to retrieve the typed letter, it would simply use the Fourier formula for the 
Fourier spectrum!” for each T-second interval to determine what each typed letter was. Figure 4.9 shows 
such a signal in the time-domain. 


11This content is available online at http: //cnx.org/content/m0043/2.17/. 
12«Fourier Series and Their Properties,” (2) http: //cnx.org/content/m0065/latest/\#complex 
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Figure 4.9: The encoding of signals via the Fourier spectrum is shown over three “periods.” In this ex- 
ample, only the third and fourth harmonics are used, as shown by the spectral magnitudes corresponding 
to each T-second interval plotted below the waveforms. Can you determine the phase of the harmonics 
from the waveform? 


In this Fourier-series encoding scheme, we have used the fact that spectral coefficients can be indepen- 
dently specified and that they can be uniquely recovered from the time-domain signal over one “period.” Do 
note that the signal representing the entire document is no longer periodic. By understanding the Fourier se- 
ries’ properties (in particular that coefficients are determined only over a T-second interval, we can construct 
a communications system. This approach represents a simplification of how modern modems represent text 
that they transmit over telephone lines. 


4.7 Filtering Periodic Signals” 


The Fourier series representation of a periodic signal makes it easy to determine how a linear, time-invariant 
filter reshapes such signals in general. The fundamental property of a linear system is that its input-output 
relation obeys superposition: S [a151 (t) + a252 (t)| = a1& [s1 (t)] + a2& [s2 (t)]. Because the Fourier series 
represents a periodic signal as a linear combination of complex exponentials, we can exploit the superposition 
property. Furthermore, we found for linear circuits that their output to a complex exponential input is just the 
frequency response evaluated at the signal’s frequency times the complex exponential. Said mathematically, 


if x(t) =e’ T , then the output y(t) = H (5 e’ T because f = r Thus, if x (¢) is periodic thereby 


having a Fourier series, a linear circuit’s output to this signal will be the superposition of the output to each 
component. 


y(t) = y ChH (5) jr (4.27) 


k=—0o 


Thus, the output has a Fourier series, which means that it too is periodic. Its Fourier coefficients equal 
Cpr z): To obtain the spectrum of the output, we simply multiply the input spectrum by 


the frequency response. The circuit modifies the magnitude and phase of each Fourier coefficient. Note 
especially that while the Fourier coefficients do not depend on the signal’s period, the circuit’s transfer 
function does depend on frequency, which means that the circuit’s output will differ as the period varies. 


13This content is available online at http: //cnx.org/content/m0044/2.10/. 
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Figure 4.10: A periodic pulse signal, such as shown on the left part (A/T = 0.2), serves as the input to 
an RC lowpass filter. The input’s period was 1 ms (millisecond). The filter’s cutoff frequency was set to 
the various values indicated in the top row, which display the output signal’s spectrum and the filter’s 
transfer function. The bottom row shows the output signal derived from the Fourier series coefficients 
shown in the top row. (a) Periodic pulse signal (b) Top plots show the pulse signal’s spectrum for various 
cutoff frequencies. Bottom plots show the filter’s output signals. 


Example 4.3 
The periodic pulse signal shown on the left above serves as the input to a RC-circuit shown in 
Figure 3.28 that has the transfer function (calculated elsewhere) 


1 


— 4.28 
1+ 727fRC ( ) 


H(f) 
Figure 4.10 shows the output changes as we vary the filter’s cutoff frequency. Note how the signal’s 
spectrum extends well above its fundamental frequency. Having a cutoff frequency ten times higher 
than the fundamental does perceptibly change the output waveform, rounding the leading and 
trailing edges. As the cutoff frequency decreases (center, then left), the rounding becomes more 
prominent, with the leftmost waveform showing a small ripple. 


Exercise 4.11 (Solution on p. 140.) 
What is the average value of each output waveform? The correct answer may surprise you. 


This example also illustrates the impact a lowpass filter can have on a waveform. The simple RC filter used 
here has a rather gradual frequency response, which means that higher harmonics are smoothly suppressed. 
Later, we will describe filters that have much more rapidly varying frequency responses, allowing a much 
more dramatic selection of the input’s Fourier coefficients. 

More importantly, we have calculated the output of a circuit to a periodic input without writing, 
much less solving, the differential equation governing the circuit’s behavior. Furthermore, we made these 
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calculations entirely in the frequency domain. Using Fourier series, we can calculate how any linear circuit 
will respond to a periodic input. 


4.8 Derivation of the Fourier Transform™ 


Fourier series clearly open the frequency domain as an interesting and useful way of determining how circuits 
and systems respond to periodic input signals. Can we use similar techniques for non-periodic signals? What 
is the response of the filter to a single pulse? Addressing these issues requires us to find the Fourier spectrum 
of all signals, both periodic and non-periodic ones. We need a definition for the Fourier spectrum of a signal, 
periodic or not. This spectrum is calculated by what is known as the Fourier transform. 

Let sr (t) be a periodic signal having period T. We want to consider what happens to this signal’s 
spectrum as we let the period become longer and longer. We denote the spectrum for any assumed value of 
the period by cz (T). We calculate the spectrum according to the familiar formula 


gee Qmkt 
cr (LT) = z/ sr(t)e ? T dt (4.29) 


where we have used a symmetric placement of the integration interval about the origin for subsequent deriva- 
tional convenience. Let f be a fixed frequency equaling k/T; we vary the frequency index k proportionally 
as we increase the period. Define 
T/2 
Sr (f) = T cz (T) = i sp (t)e 3?" *dt (4.30) 
-T/2 


making the corresponding Fourier series 


sr(t)= > Sr(fea (4.31) 


k=—0o 


As the period increases, the spectral lines become closer together, becoming a continuum. Therefore, 


jim srt) = s(t) = / _ 8 f) ei?" Taf (4.32) 
with a 
S(fy= : s(t) e 9?" Fede (4.33) 


S'(f) is the Fourier transform of s (t) (the Fourier transform is symbolically denoted by the uppercase version 
of the signal’s symbol) and is defined for any signal for which the integral ((4.33)) converges. 


Example 4.4 
Let’s calculate the Fourier transform of the pulse signal p(t). 


oo A 
RG. =| p(t etna | e Paftdt = —— (ef _ 4 
me ~oo s 0 Tpanf | ) 
_ _jnpa Sin (7 fA) 
=e ee 
uf 


Note how closely this result resembles the expression for Fourier series coefficients of the periodic 
pulse signal (4.10). 
Figure 4.11 shows how increasing the period does indeed lead to a continuum of coefficients, and that the 


n (t) 


si 
Fourier transform does correspond to what the continuum becomes. The quantity a has a special name, 


14This content is available online at http: //cnx.org/content/m0046/2.21/. 
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Figure 4.11: The upper plot shows the magnitude of the Fourier series spectrum for the case of T = 1 
with the Fourier transform of p(t) shown as a dashed line. For the bottom panel, we expanded the period 
to T = 5, keeping the pulse’s duration fixed at 0.2, and computed its Fourier series coefficients. 


the sinc (pronounced “sink”) function, and is denoted by sinc (t). Thus, the magnitude of the pulse’s Fourier 
transform equals |Asinc (7 fA) |. 

The Fourier transform relates a signal’s time and frequency domain representations to each other. The 
direct Fourier transform (or simply the Fourier transform) calculates a signal’s frequency domain repre- 
sentation from its time-domain variant (4.34). The inverse Fourier transform (4.35) finds the time-domain 
representation from the frequency domain. Rather than explicitly writing the required integral, we often 
symbolically express these transform calculations as F (s) and F~! (S$), respectively. 


F (s) =S(f) 
= / s(t) e J27 Ft dt ie) 


F-(S\=s 
7 (4.35) 


(t) 
f snerrg 
We must have s(t) = F~' (F(s(t))) and S(f) =F (F7! 


minor exceptions. 


(S(f))), and these results are indeed valid with 


NOTE: Recall that the Fourier series for a square wave gives a value for the signal at the dis- 
continuities equal to the average value of the jump. This value may differ from how the signal is 
defined in the time domain, but being unequal at a point is indeed minor. 


Showing that you “get back to where you started” is difficult from an analytic viewpoint, and we won’t try 
here. Note that the direct and inverse transforms differ only in the sign of the exponent. 
Exercise 4.12 (Solution on p. 140.) 
The differing exponent signs means that some curious results occur when we use the wrong sign. 
What is F(S(f))? In other words, use the wrong exponent sign in evaluating the inverse Fourier 
transform. 
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Properties of the Fourier transform and some useful transform pairs are provided in the accompanying tables 
(Table 4.1: Short Table of Fourier Transform Pairs and Table 4.2: Fourier Transform Properties). Especially 
important among these properties is Parseval’s Theorem, which states that power computed in either 
domain equals the power in the other. 


[ ewa= fo isinrar (4.36) 


Of practical importance is the conjugate symmetry property: When s(t) is real-valued, the spectrum at 
negative frequencies equals the complex conjugate of the spectrum at the corresponding positive frequencies. 
Consequently, we need only plot the positive frequency portion of the spectrum (we can easily determine the 
remainder of the spectrum). 


Exercise 4.13 (Solution on p. 140.) 

How many Fourier transform operations need to be applied to get the original signal back: 

F (+++ (F (s))) = s(t)? 
Note that the mathematical relationships between the time domain and frequency domain versions of the 
same signal are termed transforms. We are transforming (in the nontechnical meaning of the word) a 
signal from one representation to another. We express Fourier transform pairs as (s(t)<— S(f)). A 
signal’s time and frequency domain representations are uniquely related to each other. A signal thus “exists” 
in both the time and frequency domains, with the Fourier transform bridging between the two. We can 
define an information carrying signal in either the time or frequency domains; it behooves the wise engineer 
to use the simpler of the two. 

A common misunderstanding is that while a signal exists in both the time and frequency domains, a single 
formula expressing a signal must contain only time or frequency: Both cannot be present simultaneously. 
This situation mirrors what happens with complex amplitudes in circuits: As we reveal how communications 
systems work and are designed, we will define signals entirely in the frequency domain without explicitly find- 
ing their time domain variants. This idea is shown in Section 4.6, where we define Fourier series coefficients 
according to letter to be transmitted. Thus, a signal, though most familiarly defined in the time-domain, 
really can be defined equally as well (and sometimes more easily) in the frequency domain. For example, 
impedances depend on frequency and the time variable cannot appear. 

We will learn that finding a linear, time-invariant system’s output in the time domain can be most easily 
calculated by determining the input signal’s spectrum, performing a simple calculation in the frequency 
domain, and inverse transforming the result. Furthermore, understanding communications and information 
processing systems requires a thorough understanding of signal structure and of how systems work in both 
the time and frequency domains. 

The only difficulty in calculating the Fourier transform of any signal occurs when we have periodic signals 
(in either domain). Realizing that the Fourier series is a special case of the Fourier transform, we simply 
calculate the Fourier series coefficients instead, and plot them along with the spectra of non-periodic signals 
on the same frequency axis. 


s(t) S(f) 
1 
eat u(t) janfta 
lel 2a 
c dn? f2 + a2 
_ fi, |tl< | sin fA) 
Cs e |¢| > A nf 
sin (27Wt) _ Ji, lfl<Ww 
mt sin= {i lfl>W 


Table 4.1 


Time Domain 


Frequency Domain 


Linearity a1 8} (t) + a282 (t) | a1S) (f) + a2S2(f) 
Conjugate Symmetry s(t)ER S(f) =S8(-f)* 
Even Symmetry s(t) = s(t) S(f) =S(-f) 
Odd Symmetry S(f) =—S(-f) 


Scale Change 


Time Delay s(t—T) e F277 5 (f) 
Complex Modulation ej? fot s (t) S(f — fo) 
Amplitude Modulation by Cosine | s (¢) cos (27 fot) SUF = fo) 5 SU + fo) 
Amplitude Modulation by Sine s(t) sin (27 fot) SUF = fo) 7 SU + fo) 
J 
Differentiation is (t) jan fs (f) 
t 
Integration / 8 (a) da ae (f) if S$(0) =0 
266 jan f 
Multiplication by t ts (t) ! d sp) 
ultiplication by 8 or Tf of 
Area i s(t) dt S' (0) 
Value at Origin s (0) / S(f) df 


Parseval’s Theorem 


ie |s (t)|?dt 


[swear 


—oco 


Fourier coefficients are cz, = 


Example 4.5 
In communications, a very important operation on a signal s(t) is to amplitude modulate it. 
Using this operation more as an example rather than elaborating the communications aspects here, 
we want to compute the Fourier transform — the spectrum — of 


Table 4.2 


(1+ s(t)) cos (2m fet) 


(1+ s(t)) cos (2m fot) = cos (27 fet) + s (t) cos (27 fet) 


s(t) cos (27f.t) = [- S (f) ef? df - cos (27 fet) 


For the spectrum of cos (27 f.t), we use the Fourier series. Its period is 1/f., and its only nonzero 
3. The second term is not periodic unless s(t) has the same period 
as the sinusoid. Using Euler’s relation, the spectrum of the second term can be derived as 


117 


118 CHAPTER 4. FREQUENCY DOMAIN 


Using Euler’s relation for the cosine, 


s(t) cos (27 f,t) = sf. S(f) eft fodtar 4 sf. S(f) ef2"F-fetar 


sf su-memrap+s [site tera 


i SUP = fe) + SF fe) sonst ge 
2 


—oo 
Exploiting the uniqueness property of the Fourier transform, we have 


FiOextrioi= =” : Sf + fe) (4.37) 


This component of the spectrum consists of the original signal’s spectrum delayed and advanced in 
frequency. The spectrum of the amplitude modulated signal is shown in Figure 4.12. 
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Figure 4.12: A signal which has a triangular shaped spectrum is shown in the top plot. Its highest 
frequency — the largest frequency containing power — is W Hz. Once amplitude modulated, the resulting 
spectrum has “lines” corresponding to the Fourier series components at +f. and the original triangular 
spectrum shifted to components at +f. and scaled by s: 


Note how in this figure the signal s (£) is defined in the frequency domain. To find its time domain 
representation, we simply use the inverse Fourier transform. 


Exercise 4.14 (Solution on p. 140.) 
What is the signal s (¢) that corresponds to the spectrum shown in the upper panel of Figure 4.12? 
Exercise 4.15 (Solution on p. 140.) 


What is the power in a (t), the amplitude-modulated signal? Try the calculation in both the time 
and frequency domains. 


In this example, we call the signal s (t) a baseband signal because its power is contained at low frequencies. 
Signals such as speech and the Dow Jones averages are baseband signals. The baseband signal’s bandwidth 
equals W, the highest frequency at which it has power. Since « (t)’s spectrum is confined to a frequency band 
not close to the origin (we assume (f, >> W)), we have a bandpass signal. The bandwidth of a bandpass 
signal is not its highest frequency, but the range of positive frequencies where the signal has power. Thus, 
in this example, the bandwidth is 2WHz. Why a signal’s bandwidth should depend on its spectral shape 
will become clear once we develop communications systems. 


119 


4.9 Linear Time Invariant Systems” 


When we apply a periodic input to a linear, time-invariant system, the output is periodic and has Fourier 
series coefficients equal to the product of the system’s frequency response and the input’s Fourier coefficients 
(Filtering Periodic Signals (4.27)). The way we derived the spectrum of non-periodic signal from periodic 
ones makes it clear that the same kind of result works when the input is not periodic: If x(t) serves as 
the input to a linear, time-invariant system having frequency response H (f), the spectrum of 
the output is X (f) H (f). 


Example 4.6 

Let’s use this frequency-domain input-output relationship for linear, time-invariant systems to find 
a formula for the RC-circuit’s response to a pulse input. We have expressions for the input’s 
spectrum and the system’s frequency response. 


P(f)= corel) (4.38) 
H(f)= ESP (4.39) 


Thus, the output’s Fourier transform equals 


asin (afA) 1 
wf 1+ 9207fRC 


You won't find this Fourier transform in our table, and the required integral is difficult to evaluate 
as the expression stands. This situation requires cleverness and an understanding of the Fourier 
transform’s properties. In particular, recall Euler’s relation for the sinusoidal term and note the 
fact that multiplication by a complex exponential in the frequency domain amounts to a time delay. 
Let’s momentarily make the expression for Y (f) . 


sin (7 fA) _jnpa O™t® —eFttA 
=e ——— 


Y (f) =e IF (4.40) 


enon fh 
nf janf (4.41) 
1 
an Ks —janfa 
core ) 
Consequently, 
1 . it 

¥ =e pe A) = 4.42 
(f) Fant | ) [jan FRC ee) 


The table of Fourier transform properties (Table 4.2) suggests thinking about this expression as a 
product of terms. 


1 
e Multiplication by —— means integration. 


jan f wee 
e Multiplication by the complex exponential e~J?7/4 means delay by A seconds in the time 
domain. 
e The term 1 — e~/?7/4 means, in the time domain, subtract the time-delayed signal from its 
original. 


1 
e The inverse transform of the frequency response is yaa FO a(t). 


We can translate each of these frequency-domain products into time-domain operations in any 
order we like because the order in which multiplications occur doesn’t affect the result. Let’s 
start with the product of 1/j27f (integration in the time domain) and the transfer function: 


1 1 
Ve | ae 4.4 
jinf 1+j2nfRC - > ( - ) a(t ee) 


15This content is available online at http: //cnx.org/content/m0048/2.18/. 
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The middle term in the expression for Y (f) consists of the difference of two terms: the constant 
1 and the complex exponential e~/?"/4. Because of the Fourier transform’s linearity, we simply 
subtract the results. 


¥ (f) > (1 eFC) uit) — (1 e ARC) w(t — A) (4.44) 


Note that in delaying the signal how we carefully included the unit step. The second term in this 
result does not begin until t = A. Thus, the waveforms shown in the Filtering Periodic Signals 
example (Figure 4.10) mentioned above are exponentials. We say that the time constant of an 
exponentially decaying signal equals the time it takes to decrease by 1/e of its original value. Thus, 
the time-constant of the rising and falling portions of the output equal the product of the circuit’s 
resistance and capacitance. 


Exercise 4.16 (Solution on p. 140.) 
Derive the filter’s output by considering the terms in (4.41) in the order given. Integrate last rather 
than first. You should get the same answer. 


In this example, we used the table extensively to find the inverse Fourier transform, relying mostly on 


—j2nrfa 


what multiplication by certain factors, like - and e , meant. We essentially treated multiplication 


j20 
by these factors as if they were transfer functions of some fictitious circuit. The transfer function 


1 
j2af 
corresponded to a circuit that integrated, and e~/?7/4 to one that delayed. We even implicitly interpreted 
the circuit’s transfer function as the input’s spectrum! This approach to finding inverse transforms — breaking 
down a complicated expression into products and sums of simple components — is the engineer’s way of 
breaking down the problem into several subproblems that are much easier to solve and then gluing the results 
together. Along the way we may make the system serve as the input, but in the rule Y (f) = X (f) H(f), 
which term is the input and which is the transfer function is merely a notational matter (we labeled one 
factor with an X and the other with an H). 


4.9.1 Transfer Functions 


The notion of a transfer function applies well beyond linear circuits. Although we don’t have all we need 
to demonstrate the result as yet, all linear, time-invariant systems have a frequency-domain input-output 
relation given by the product of the input’s Fourier transform and the system’s transfer function. Thus, 
linear circuits are a special case of linear, time-invariant systems. As we tackle more sophisticated problems 
in transmitting, manipulating, and receiving information, we will assume linear systems having certain 
properties (transfer functions) without worrying about what circuit has the desired property. At this point, 
you may be concerned that this approach is glib, and rightly so. Later we’ll show that by involving software 
that we really don’t need to be concerned about constructing a transfer function from circuit elements and 
op-amps. 


4.9.2 Commutative Transfer Functions 


Another interesting notion arises from the commutative property of multiplication (exploited in an example 
above (Example 4.6)): We can arbitrarily choose an order in which to apply each product. Consider a 
cascade of two linear, time-invariant systems. Because the Fourier transform of the first system’s output is 
X (f) H, (f) and it serves as the second system’s input, the cascade’s output spectrum is X (f) Hy (f) He (f). 
Because this product also equals X (f) H2(f) Hi (f), the cascade having the linear systems in the 
opposite order yields the same result. Furthermore, the cascade acts like a single linear system, 
having transfer function H, (f) H2(f). This result applies to other configurations of linear, time-invariant 
systems as well; see this Frequency Domain Problem (Problem 4.13). Engineers exploit this property by 
determining what transfer function they want, then breaking it down into components arranged according to 
standard configurations. Using the fact that op-amp circuits can be connected in cascade with the transfer 
function equaling the product of its component’s transfer function (see this analog signal processing problem 
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Figure 4.13: The vocal tract is shown in cross-section. Air pressure produced by the lungs forces air 
through the vocal cords that, when under tension, produce puffs of air that excite resonances in the vocal 
and nasal cavities. What are not shown are the brain and the musculature that control the entire speech 
production process. 


(Problem 3.43)), we find a ready way of realizing designs. We now understand why op-amp implementations 
of transfer functions are so important. 


4.10 Modeling the Speech Signal” 


The information contained in the spoken word is conveyed by the speech signal. Because we shall analyze 
several speech transmission and processing schemes, we need to understand the speech signal’s structure 
— what’s special about the speech signal — and how we can describe and model speech production. This 
modeling effort consists of finding a system’s description of how relatively unstructured signals, arising from 
simple sources, are given structure by passing them through an interconnection of systems to yield speech. 
For speech and for many other situations, system choice is governed by the physics underlying the actual 


16This content is available online at http: //cnx.org/content/m0049/2.27/. 


122 CHAPTER 4. FREQUENCY DOMAIN 


neural 
control control 
1 
1 


! Mt) | Yocar ]'PT! Vocar | st 
Lungs > Cords i > Tract > 


neural 


Figure 4.14: The systems model for the vocal tract. The signals /(t), pr(t), and s(t) are the air 
pressure provided by the lungs, the periodic pulse output provided by the vocal cords, and the speech 
output respectively. Control signals from the brain are shown as entering the systems from the top. 
Clearly, these come from the same source, but for modeling purposes we describe them separately since 
they control different aspects of the speech signal. 


production process. Because the fundamental equation of acoustics — the wave equation — applies here and 
is linear, we can use linear systems in our model with a fair amount of accuracy. The naturalness of linear 
system models for speech does not extend to other situations. In many cases, the underlying mathematics 
governed by the physics, biology, and/or chemistry of the problem are nonlinear, leaving linear systems 
models as approximations. Nonlinear models are far more difficult at the current state of knowledge to 
understand, and information engineers frequently prefer linear models because they provide a greater level 
of comfort, but not necessarily a sufficient level of accuracy. 

Figure 4.13 shows the actual speech production system and Figure 4.14 the model speech production 
system. The characteristics of the model depends on whether you are saying a vowel or a consonant. We 
concentrate first on the vowel production mechanism. When the vocal cords are placed under tension by 
the surrounding musculature, air pressure from the lungs causes the vocal cords to vibrate. To visualize this 
effect, take a rubber band and hold it in front of your lips. If held open when you blow through it, the air 
passes through more or less freely; this situation corresponds to “breathing mode.” If held tautly and close 
together, blowing through the opening causes the sides of the rubber band to vibrate. This effect works best 
with a wide rubber band. You can imagine what the airflow is like on the opposite side of the rubber band 
or the vocal cords. Your lung power is the simple source referred to earlier; it can be modeled as a constant 
supply of air pressure. The vocal cords respond to this input by vibrating, which means the output of this 
system is some periodic function. 


Exercise 4.17 (Solution on p. 140.) 
Note that the vocal cord system takes a constant input and produces a periodic airflow that corre- 
sponds to its output signal. Is this system linear or nonlinear? Justify your answer. 


Singers modify vocal cord tension to change the pitch to produce the desired musical note. Vocal cord 
tension is governed by a control input to the musculature; in system’s models we represent control inputs as 
signals coming into the top or bottom of the system. Certainly in the case of speech and in many other cases 
as well, it is the control input that carries information, impressing it on the system’s output. The change of 
signal structure resulting from varying the control input enables information to be conveyed by the signal, 
a process generically known as modulation. In singing, musicality is largely conveyed by pitch; in western 
speech, pitch is much less important. A sentence can be read in a monotone fashion without completely 
destroying the information expressed by the sentence. However, the difference between a statement and a 
question is frequently expressed by pitch changes. For example, note the sound differences between “Let’s 
go to the park” and “Let’s go to the park?”; 

For some consonants, the vocal cords vibrate just as in vowels. For example, the so-called nasal sounds 
‘“n” and “m” have this property. For others, the vocal cords do not produce a periodic output. Going back to 
mechanism, when consonants such as “f” are produced, the vocal cords are placed under much less tension, 
which results in turbulent flow. The resulting output airflow is quite erratic, so much so that we describe it 
as being noise. We define noise carefully later when we delve into communication problems. 

The vocal cords’ periodic output can be well described by the periodic pulse train pr (t) as shown in 
the periodic pulse signal (Figure 4.1), with T’ denoting the pitch period. The spectrum of this signal (4.9) 
contains harmonics of the frequency 1/T, what is known as the pitch frequency or the fundamental 
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Figure 4.15: The ideal frequency response of the vocal tract as it produces the sounds “oh” and “ee” 
are shown on the top left and top right, respectively. The spectral peaks are known as formants, and 
are numbered consecutively from low to high frequency. The bottom plots show speech waveforms 
corresponding to these sounds. 


frequency FO. The primary difference between adult male and female/prepubescent speech is pitch. Before 
puberty, pitch frequency for normal speech ranges between 150-400 Hz for both males and females. After 
puberty, the vocal cords of males undergo a physical change, which has the effect of lowering their pitch 
frequency to the range 80-160 Hz. If we could examine the vocal cord output, we could probably discern 
whether the speaker was male or female. This difference is also readily apparent in the speech signal itself. 

To simplify our speech modeling effort, we shall assume that the pitch period is constant. With this 
simplification, we collapse the vocal-cord-lung system as a simple source that produces the periodic pulse 
signal (Figure 4.14). The sound pressure signal thus produced enters the mouth behind the tongue, creates 
acoustic disturbances, and exits primarily through the lips and to some extent through the nose. Speech 
specialists tend to name the mouth, tongue, teeth, lips, and nasal cavity the vocal tract. The physics 
governing the sound disturbances produced in the vocal tract and those of an organ pipe are quite similar. 
Whereas the organ pipe has the simple physical structure of a straight tube, the cross-section of the vocal 
tract “tube” varies along its length because of the positions of the tongue, teeth, and lips. It is these positions 
that are controlled by the brain to produce the vowel sounds. Spreading the lips, bringing the teeth together, 
and bringing the tongue toward the front portion of the roof of the mouth produces the sound “ee.” Rounding 
the lips, spreading the teeth, and positioning the tongue toward the back of the oral cavity produces the 
sound “oh.” These variations result in a linear, time-invariant system that has a frequency response typified 
by several peaks, as shown in Figure 4.15. 

These peaks are known as formants. Thus, speech signal processors would say that the sound “oh” has 
a higher first formant frequency than the sound “ee,” with F2 being much higher during “ee.” F2 and F3 
(the second and third formants) have more energy in “ee” than in “oh.” Rather than serving as a filter, 
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rejecting high or low frequencies, the vocal tract serves to shape the spectrum of the vocal cords. In the 
time domain, we have a periodic signal, the pitch, serving as the input to a linear system. We know that 
the output—the speech signal we utter and that is heard by others and ourselves—will also be periodic. 
Example time-domain speech signals are shown in Figure 4.15, where the periodicity is quite apparent. 


Exercise 4.18 (Solution on p. 140.) 
From the waveform plots shown in Figure 4.15, determine the pitch period and the pitch frequency. 


Since speech signals are periodic, speech has a Fourier series representation given by a linear circuit’s response 
to a periodic signal (4.27). Because the acoustics of the vocal tract are linear, we know that the spectrum of 
the output equals the product of the pitch signal’s spectrum and the vocal tract’s frequency response. We 
thus obtain the fundamental model of speech production. 


S(f) = Pr (f) Av (f) (4.45) 


Here, Hy (f) is the transfer function of the vocal tract system. The Fourier series for the vocal cords’ output, 
derived in this equation (p. 100), is 
. (rk 
ae) 


Ck = Ae T a 


and is plotted on the top in Figure 4.16. If we had, for example, a male speaker with about a 110 Hz 
pitch (T+ 9.1ms) saying the vowel “oh,” the spectrum of his speech predicted by our model is shown in 
Figure 4.16(b). 

The model spectrum idealizes the measured spectrum, and captures all the important features. The 
measured spectrum certainly demonstrates what are known as pitch lines, and we realize from our model 
that they are due to the vocal cord’s periodic excitation of the vocal tract. The vocal tract’s shaping of the 
line spectrum is clearly evident, but difficult to discern exactly, especially at the higher frequencies. The 
model transfer function for the vocal tract makes the formants much more readily evident. 


Exercise 4.19 (Solution on p. 141.) 
The Fourier series coefficients for speech are related to the vocal tract’s transfer function only at 
the frequencies k/T, k € {1,2,...}; see previous result (4.9). Would male or female speech tend to 
have a more clearly identifiable formant structure when its spectrum is computed? Consider, for 
example, how the spectrum shown on the right in Figure 4.16 would change if the pitch were twice 
as high (~ 300Hz). 
When we speak, pitch and the vocal tract’s transfer function are not static; they change according to their 
control signals to produce speech. Engineers typically display how the speech spectrum changes over time 
with what is known as a spectrogram (detailed in Section 5.10), an example of which is shown in Figure 4.17. 
Note how the line spectrum, which indicates how the pitch changes, is visible during the vowels, but not 
during the consonants (like the ce in “Rice”). 

The fundamental model for speech indicates how engineers use the physics underlying the signal gen- 
eration process and exploit its structure to produce a systems model that suppresses the physics while 
emphasizing how the signal is “constructed.” From everyday life, we know that speech contains a wealth of 
information. We want to determine how to transmit and receive it. Efficient and effective speech transmis- 
sion requires us to know the signal’s properties and its structure (as expressed by the fundamental model of 
speech production). We see from Figure 4.17, for example, that speech contains significant energy from zero 
frequency up to around 5 kHz. 

Effective speech transmission systems must be able to cope with signals having this bandwidth. It 
is interesting that one system that does not support this 5 kHz bandwidth is the telephone: Telephone 
systems act like a bandpass filter passing energy between about 200 Hz and 3.2 kHz. The most important 
consequence of this filtering is the removal of high frequency energy. In our sample utterance, the “ce” sound 
in “Rice” contains most of its energy above 3.2 kHz; this filtering effect is why it is extremely difficult to 
distinguish the sounds “s” and “f” over the telephone. Try this yourself: Call a friend and determine if they 
can distinguish between the words “six” and “fix.” If you say these words in isolation so that no context 
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Figure 4.16: The vocal tract’s transfer function, shown as the thin, smooth line, is superimposed on 
the spectrum of actual male speech corresponding to the sound “oh.” The pitch lines corresponding to 
harmonics of the pitch frequency are indicated. (a) The vocal cords’ output spectrum Pr (f). (b) The 
vocal tract’s transfer function, Hy (f) and the speech spectrum. 


provides a hint about which word you are saying, your friend will not be able to tell them apart. Radio does 
support this bandwidth (see more about AM and FM radio systems in Section 6.11). 

Efficient speech transmission systems exploit the speech signal’s special structure: What makes speech 
speech? You can conjure many signals that span the same frequencies as speech—car engine sounds, violin 
music, dog barks—but don’t sound at all like speech. We shall learn later that transmission of any 5 kHz 
bandwidth signal requires about 80 kbps (thousands of bits per second) to transmit digitally. Speech signals 
can be transmitted using less than 1 kbps because of its special structure. To reduce the “digital bandwidth” 
so drastically means that engineers spent many years to develop signal processing and coding methods that 
could capture the special characteristics of speech without destroying how it sounds. If you used a speech 
transmission system to send a violin sound, it would arrive horribly distorted; speech transmitted the same 
way would sound fine. 

Exploiting the special structure of speech requires going beyond the capabilities of analog signal processing 
systems. Many speech transmission systems work by finding the speaker’s pitch and the formant frequencies. 
Fundamentally, we need to do more than filtering to determine the speech signal’s structure; we need to 
manipulate signals in more ways than are possible with analog systems. Such flexibility is achievable (but 
not without some loss) with programmable digital systems. 
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Figure 4.17: Displayed is the spectrogram of the author saying “Rice University.” Blue indicates low 
energy portion of the spectrum, with red indicating the most energetic portions. Below the spectrogram 
is the time-domain speech signal, where the periodicities can be seen. 


4.11 Frequency Domain Problems” 

Problem 4.1: Simple Fourier Series 

Find the Fourier series representations of the following signals without explicitly calculating Fourier integrals. 
What is the signal’s period in each case? 


(a) s(t) = sin (t) (b) s(t) = sin? (t) 
(c) s(t) = cos (¢) + 2cos (2t) (d) s(t) = cos (2t) cos (t) 
(e) s(t) = cos (10nt + 7) (1+cos(2nt)) (f) s(t) given by the waveform depicted in Figure 4.18. 


Problem 4.2: Fourier Series 

Find the Fourier series representation for the periodic signals shown in Figure 4.19. For the third signal, 
find the complex Fourier series for the triangle wave without performing the usual Fourier integrals. Hint: 
How is this signal related to one for which you already have the series? 


Problem 4.3: Phase Distortion 
We can learn about phase distortion by returning to circuits and investigate the circuit shown in Figure 4.20. 


17This content is available online at http: //cnx.org/content/m10350/2.38/. 
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(a) Find this filter’s transfer function. 

(b) Find the magnitude and phase of this transfer function. How would you characterize this circuit? 

(c) Let v;, (t) be a square-wave of period T. What is the Fourier series for the output voltage? 

(d) Use Matlab to find the output’s waveform for the cases T = 0.01 and T = 2. What value of T delineates 
the two kinds of results you found? The software in fourier2.m might be useful. 

(e) Instead of the depicted circuit, the square wave is passed through a system that delays its input, which 
applies a linear phase shift to the signal’s spectrum. Let the delay 7 be f. Use the transfer function 
of a delay to compute using Matlab the Fourier series of the output. Show that the square wave is 
indeed delayed. 


Problem 4.4: Approximating Periodic Signals 
Often, we want to approximate a reference signal by a somewhat simpler signal. To assess the quality of 
an approximation, the most frequently used error measure is the mean-squared error. For a periodic signal 


s(t), : 
ge a (s(t) — 3 (t))2at 


where s(t) is the reference signal and § (t) its approximation. One convenient way of finding approximations 
for periodic signals is to truncate their Fourier series. 


The point of this problem is to analyze whether this approach is the best (i.e., always minimizes the mean- 
squared error). 


(a) Find a frequency-domain expression for the approximation error when we use the truncated Fourier 
series as the approximation. 

(b) Instead of truncating the series, let’s generalize the nature of the approximation to including any set of 
2K +1 terms: We'll always include the co and the negative indexed term corresponding to cx. What 
selection of terms minimizes the mean-squared error? Find an expression for the mean-squared error 
resulting from your choice. 

(c) Find the Fourier series for the signal depicted in Figure 4.21. Use Matlab to find the truncated 
approximation and best approximation involving two terms. Plot the mean-squared error as a function 
of K for both approximations. 


Figure 4.21 


Problem 4.5: Long, Hot Days 

The daily temperature is a consequence of several effects, one of them being the sun’s heating. If this were 
the dominant effect, then daily temperatures would be proportional to the number of daylight hours. The 
map shows that the hottest day of the year varies around the country, but does not occur on the longest day 
of the year: June 20-21 (except in the desert Southwest). The plot details the average daily high temperature 
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Figure 4.22 


in Houston. In this problem, we want to understand the temperature component of our environment using 
Fourier series and linear system theory. The file temperature.mat contains these data (daylight hours in 
the variable daylight, average high temperature in high) for Houston, Texas. 


(a) Let the length of day serve as the sole input to a system having an output equal to the average daily 
temperature. Examining the plots of input and output, would you say that the system is linear or not? 
How did you reach you conclusion? 

(b) Find the first five terms (co,...,c4) of the Fourier series for each signal using MATLAB. Approximate 


130 CHAPTER 4. FREQUENCY DOMAIN 


the integral needed to calculate Fourier series coefficients by a Riemann sum as follows. 


365 
S- s(n)e—F27nk/366 fp — 0, ...,4 


n=0 


1 
Ck = san 
"366 
(c) What is the harmonic distortion in the two signals? Exclude cg from this calculation. 
(d) Because the harmonic distortion is small, let’s concentrate only on the first harmonic. What is the 
phase shift between input and output signals, expressed both in degrees and weeks? 


This phase shift is very important in developing a simple circuit model for the system that describes how 
daily temperatures are related to the number of daylight hours. For example, because there is a phase shift, 
we know that a simple gain (a resistive circuit) won’t work. 


(e) Find the transfer function of the simplest possible linear model (a first-order filter, either lowpass or 
highpass) that would describe the data. Note that the phase shift is negative: the output lags (occurs 
later) than the input. Based on this observation, which first-order model fits the data and what is its 
transfer function? 

(f) Characterize and interpret the structure of this model. Here, let the input (length of day) be a voltage 
source and the output (daily high) a voltage. Give a physical explanation for the phase shift. 


Problem 4.6: Fourier Transform Pairs 
Find the Fourier or inverse Fourier transform of the following. 


(a) x(t) = el (b) x(t) = te~™ u(t) 
— I fl < Ww — p—at 
(c) X(f) = ae (a) 2 (1) = e-** cos (20 fot) u(t) 


Problem 4.7: Duality in Fourier Transforms 
“Duality” means that the Fourier transform and the inverse Fourier transform are very similar. Consequently, 
the waveform s (t) in the time domain and the spectrum s (f) have a Fourier transform and an inverse Fourier 
transform, respectively, that are very similar. 


(a) Calculate the Fourier transform of the signal shown below (Figure 4.23(a)). 

(b) Calculate the inverse Fourier transform of the spectrum shown below (Figure 4.23(b)). 

(c) How are these answers related? What is the general relationship between the Fourier transform of s (t) 
and the inverse transform of s (f)? 


(a) (b) 
Figure 4.23 


Problem 4.8: Spectra of Pulse Sequences 
Pulse sequences occur often in digital communication and in other fields as well. What are their spectral 
properties? 


(a) Calculate the Fourier transform of the single pulse shown below (Figure 4.24(a)). 
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(b) Calculate the Fourier transform of the two-pulse sequence shown below (Figure 4.24(b)). 

(c) Calculate the Fourier transform for the ten-pulse sequence shown in below (Figure 4.24(c)). You 
should look for a general expression that holds for sequences of any length. 

(d) Using Matlab, plot the magnitudes of the three spectra. Describe how the spectra change as the 
number of repeated pulses increases. 
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Figure 4.24 


Problem 4.9: Spectra of Digital Communication Signals 

One way to represent bits with signals is shown in Figure 4.25. If the value of a bit is a “1,” it is represented 
by a positive pulse of duration T. If it is a “0,” it is represented by a negative pulse of the same duration. 
To represent a sequence of bits, the appropriately chosen pulses are placed one after the other. 


(a) What is the spectrum of the waveform that represents the alternating bit sequence “...01010101...?” 

(b) This signal’s bandwidth is defined to be the frequency range over which 90% of the power is contained. 
What is this signal’s bandwidth? 

(c) Suppose the bit sequence becomes “...00110011....” Now what is the bandwidth? 


Problem 4.10: Lowpass Filtering a Square Wave 
Let a square wave (period T’) serve as the input to a first-order lowpass system constructed as a RC filter. 
We want to derive an expression for the time-domain response of the filter to this input. 


(a) First, consider the response of the filter to a simple pulse, having unit amplitude and width z. Derive 
an expression for the filter’s output to this pulse. 

(b) Noting that the square wave is a superposition of a sequence of these pulses, what is the filter’s response 
to the square wave? 

(c) The nature of this response should change as the relation between the square wave’s period and the 


filter’s cutoff frequency change. How long must the period be so that the response does not achieve 
A | ” any 
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Figure 4.25 
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a relatively constant value between transitions in the square wave? What is the relation of the filter’s 
cutoff frequency to the square wave’s spectrum in this case? 


Problem 4.11: Mathematics with Circuits 

Simple circuits can implement simple mathematical operations, such as integration and differentiation. We 
want to develop an active circuit (it contains an op-amp) having an output that is proportional to the 
integral of its input. For example, you could use an integrator in a car to determine distance traveled from 
the speedometer. 


(a) What is the transfer function of an integrator? 
(b) Find an op-amp circuit so that its voltage output is proportional to the integral of its input for all 
signals. 


Problem 4.12: Where is that sound coming from? 

We determine where sound is coming from because we have two ears and a brain. Sound travels at a 
relatively slow speed and our brain uses the fact that sound will arrive at one ear before the other. As shown 
in Figure 4.26, a sound coming from the right arrives at the left ear 7 seconds after it arrives at the right 
ear. 

Once the brain finds this propagation delay, it can determine the sound direction. In an attempt to 
model what the brain might do, RU signal processors want to design an optimal system that delays each 
ear’s signal by some amount then adds them together. A; and A,. are the delays applied to the left and right 
signals respectively. The idea is to determine the delay values according to some criterion that is based on 
what is measured by the two ears. 


(a) What is the transfer function between the sound signal s(t) and the processor output y (t 
(b) One way of determining the delay 7 is to choose A; and A,. to maximize the power in y ( 
these maximum-power processing delays related to T? 


)? 
t). How are 


Problem 4.13: Arrangements of Systems 
Architecting a system of modular components means arranging them in various configurations to achieve 
some overall input-output relation. For each of the configurations shown in Figure 4.27, determine the overall 
transfer function between x (t) and y (t). 

The overall transfer function for the cascade (first depicted system) is particularly interesting. What 
does it say about the effect of the ordering of linear, time-invariant systems in a cascade? 


Problem 4.14: Filtering 
sin (mt) 


Let the signal s(t) = 
T 
shown in Figure 4.28. Find the expression for y (¢), the filter’s output. 


be the input to a linear, time-invariant filter having the transfer function 
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Figure 4.28 


Problem 4.15: Circuits Filter! 
A unit-amplitude pulse with duration of one second serves as the input to an RC-circuit having transfer 


function nf 
20 
iol = 
(f) 4+ jaf 


(a) How would you categorize this transfer function: lowpass, highpass, bandpass, other? 
(b) Find a circuit that corresponds to this transfer function. 
(c) Find an expression for the filter’s output. 


Problem 4.16: Reverberation 
Reverberation corresponds to adding to a signal its delayed version. 


(a) Assuming 7 represents the delay, what is the input-output relation for a reverberation system? Is 
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the system linear and time-invariant? If so, find the transfer function; if not, what linearity or time- 
invariance criterion does reverberation violate. 

(b) A music group known as the ROwls is having trouble selling its recordings. The record company’s 
engineer gets the idea of applying different delay to the low and high frequencies and adding the result 
to create a new musical effect. Thus, the ROwls’ audio would be separated into two parts (one less 
than the frequency fo, the other greater than fo), these would be delayed by 7; and 7), respectively, 
and the resulting signals added. Draw a block diagram for this new audio processing system, showing 
its various components. 

(c) How does the magnitude of the system’s transfer function depend on the two delays? 


Problem 4.17: Echoes in Telephone Systems 

A frequently encountered problem in telephones is echo. Here, because of acoustic coupling between the ear 
piece and microphone in the handset, what you hear is also sent to the person talking. That person thus 
not only hears you, but also hears her own speech delayed (because of propagation delay over the telephone 
network) and attenuated (the acoustic coupling gain is less than one). Furthermore, the same problem 
applies to you as well: The acoustic coupling occurs in her handset as well as yours. 


(a) Develop a block diagram that describes this situation. 

(b) Find the transfer function between your voice and what the listener hears. 

(c) Each telephone contains a system for reducing echoes using electrical means. What simple system 
could null the echoes? 


Problem 4.18: Effective Drug Delivery 

In most patients, it takes time for the concentration of an administered drug to achieve a constant level 
in the blood stream. Typically, if the drug concentration in the patient’s intravenous line is Cg u(t), the 
concentration in the patient’s blood stream is C;, (1 — e~**) u(t). 


(a) Assuming the relationship between drug concentration in the patient’s drug and the delivered concen- 
tration can be described as a linear, time-invariant system, what is the transfer function? 

(b) Sometimes, the drug delivery system goes awry and delivers drugs with little control. What would the 
patient’s drug concentration be if the delivered concentration were a ramp? More precisely, if it were 
Cat u(t)? 

(c) A clever doctor wants to have the flexibility to slow down or speed up the patient’s drug concentration. 
In other words, the concentration is to be C, (1—e~"') u(t), with b bigger or smaller than a. How 
should the delivered drug concentration signal be changed to achieve this concentration profile? 


Problem 4.19: Catching Speeders with Radar 

RU Electronics has been contracted to design a Doppler radar system. Radar transmitters emit a signal that 
bounces off any conducting object. Signal differences between what is sent and the radar return is processed 
and features of interest extracted. In Doppler systems, the object’s speed along the direction of the radar 
beam is the feature the design must extract. The transmitted signal is a sinusoid: x (t) = Acos (27f.t). The 
measured return signal equals B cos (20 ((fe + Af) t+ ~)), where the Doppler offset frequency Af equals 
10v, where v is the car’s velocity coming toward the transmitter. 


(a) Design a system that uses the transmitted and return signals as inputs and produces Af. 

(b) One problem with designs based on overly simplistic design goals is that they are sensitive to unmodeled 
assumptions. How would you change your design, if at all, so that whether the car is going away or 
toward the transmitter could be determined? 

(c) Suppose two objects traveling different speeds provide returns. How would you change your design, if 
at all, to accommodate multiple returns? 


Problem 4.20: Demodulating an AM Signal 
Let m (t) denote the signal that has been amplitude modulated. 


x(t) = A(1+m(t)) sin (27 fet) 
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Figure 4.29 


Radio stations try to restrict the amplitude of the signal m (t) so that it is less than one in magnitude. The 
frequency f, is very large compared to the frequency content of the signal. What we are concerned about 
here is not transmission, but reception. 


(a) The so-called coherent demodulator simply multiplies the signal x (¢) by a sinusoid having the same 
frequency as the carrier and lowpass filters the result. Analyze this receiver and show that it works. 
Assume the lowpass filter is ideal. 

(b) One issue in coherent reception is the phase of the sinusoid used by the receiver relative to that used 
by the transmitter. Assuming that the sinusoid of the receiver has a phase ¢, how does the output 
depend on ¢? What is the worst possible value for this phase? 

(c) The incoherent receiver is more commonly used because of the phase sensitivity problem inherent in 
coherent reception. Here, the receiver full-wave rectifies the received signal and lowpass filters the 
result (again ideally). Analyze this receiver. Does its output differ from that of the coherent receiver 
in a significant way? 


Problem 4.21: Unusual Amplitude Modulation 

We want to send a band-limited signal having the depicted spectrum (Figure 4.29(a)) using amplitude 
modulation. I.B. Different suggests using the square-wave carrier shown below (Figure 4.29(b)). Well, it is 
different, but his friends wonder if any technique can demodulate it. 


(a) Find an expression for X (f), the Fourier transform of the modulated signal. 

(b) Sketch the magnitude of X (f), being careful to label important magnitudes and frequencies. 

(c) What demodulation technique obviously works? 

(d) I.B. challenges three of his friends to demodulate x (¢) some other way. One friend suggests modulating 
a(t) with cos (%), another wants to try modulating with cos (zt) and the third thinks cos (#34) will 
work. Sketch the magnitude of the Fourier transform of the signal each student’s approach produces. 


Which student comes closest to recovering the original signal? Why? 


Problem 4.22: Sammy Falls Asleep... 

While sitting in ELEC 241 class, he falls asleep during a critical time when an AM receiver is being described. 
The received signal has the form r(t) = A(1 + m(t)) cos (2a f,t + ¢) where the phase ¢ is unknown. The 
message signal is m(t); it has a bandwidth of W Hz and a magnitude less than 1 (|m(t)| < 1). The phase ¢ 
is unknown. The instructor drew a diagram (Figure 4.30) for a receiver on the board; Sammy slept through 
the description of what the unknown systems where. 


(a) What are the signals x, (t) and x, (t)? 
(b) What would you put in for the unknown systems that would guarantee that the final output contained 
the message regardless of the phase? 


Hint: Think of a trigonometric identity that would prove useful. 


(c) Sammy may have been asleep, but he can think of a far simpler receiver. What is it? 
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Problem 4.23: Jamming 

Sid Richardson college decides to set up its own AM radio station KSRR. The resident electrical engineer 
decides that she can choose any carrier frequency and message bandwidth for the station. A rival college 
decides to jam its transmissions by transmitting a high-power signal that interferes with radios that try to 
receive KSRR. The jamming signal jam (t) is what is known as a sawtooth wave (depicted in Figure 4.31) 
having a period known to KSRR’s engineer. 


jam(t) 
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Figure 4.31 


(a) Find the spectrum of the jamming signal. 

(b) Can KSRR entirely circumvent the attempt to jam it by carefully choosing its carrier frequency and 
transmission bandwidth? If so, find the station’s carrier frequency and transmission bandwidth in 
terms of T,, the period of the jamming signal; if not, show why not. 


Problem 4.24: AM Stereo 

A stereophonic signal consists of a “left” signal J (t) and a “right” signal r (t) that conveys sounds coming from 
an orchestra’s left and right sides, respectively. To transmit these two signals simultaneously, the transmitter 
first forms the sum signal s+ (t) = /(t) + r(¢) and the difference signal s_ (t) = I(t) — r(t). Then, the 
transmitter amplitude-modulates the difference signal with a sinusoid having frequency 2W, where W is the 
bandwidth of the left and right signals. The sum signal and the modulated difference signal are added, the 
sum amplitude-modulated to the radio station’s carrier frequency f,, and transmitted. Assume the spectra 
of the left and right signals are as shown in Figure 4.32. 


L(f) R(f) 


Figure 4.32 


(a) What is the expression for the transmitted signal? Sketch its spectrum. 
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(b) Show the block diagram of a stereo AM receiver that can yield the left and right signals as separate 
outputs. 

(c) What signal would be produced by a conventional coherent AM receiver that expects to receive a 
standard AM signal conveying a message signal having bandwidth W? 


Problem 4.25: Novel AM Stereo Method 

A clever engineer has submitted a patent for a new method for transmitting two signals simultaneously 
in the same transmission bandwidth as commercial AM radio. As shown in Figure 4.33, her approach is to 
modulate the positive portion of the carrier with one signal and the negative portion with a second. In detail 


Example Transmitter Waveform 
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Figure 4.33 


the two message signals m, (t) and mz (t) are bandlimited to W Hz and have maximal amplitudes equal to 
1. The carrier has a frequency f. much greater than W. The transmitted signal x (t) is given by 


(t) = A(1+amy, (t))sin(27f-t) if sin (27 f.t) > 0 
7) A(1 + ama (t))sin(Qrfet) if sin(Qrf.t) < 0° 


In all cases, 0 < a < 1. The plot shows the transmitted signal when the messages are sinusoids: 
my, (t) = sin (27 fmt) and mg (t) = sin (272 fmt) where 2fm < W. You, as the patent examiner, must de- 
termine whether the scheme meets its claims and is useful. 


(a) Provide a more concise expression for the transmitted signal x (¢) than given above. 

(b) What is the receiver for this scheme? It would yield both m, (t) and mg (t) from z (¢). 

(c) Find the spectrum of the positive portion of the transmitted signal. 

(d) Determine whether this scheme satisfies the design criteria, allowing you to grant the patent. Explain 


your reasoning. 


Problem 4.26: A Radical Radio Idea 
An ELEC 241 student has the bright idea of using a square wave instead of a sinusoid as an AM carrier. 


The transmitted signal would have the form 
a(t) = A(1+m(t)) sq (t) 


where the message signal m (t) would be amplitude-limited: |m (t)| <1 
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(a) Assuming the message signal is lowpass and has a bandwidth of W Hz, what values for the square 
wave’s period T are feasible. In other words, do some combinations of W and T prevent reception? 

(b) Assuming reception is possible, can standard radios receive this innovative AM transmission? If so, 
show how a coherent receiver could demodulate it; if not, show how the coherent receiver’s output 
would be corrupted. Assume that the message bandwidth W = 5 kHz. 


Problem 4.27: Secret Communication 
An amplitude-modulated secret message m (t) has the following form. 


r(t) = A(1+m (t)) - cos(2(fe + fo)t) 


The message signal has a bandwidth of W Hz and a magnitude less than 1 (|m(t)| < 1). The idea is to 
offset the carrier frequency by fo Hz from standard radio carrier frequencies. Thus, “off-the-shelf” coherent 
demodulators would assume the carrier frequency has f, Hz. Here, fo < W. 


(a) Sketch the spectrum of the demodulated signal produced by a coherent demodulator tuned to f, Hz. 
(b) Will this demodulated signal be a “scrambled” version of the original? If so, how so; if not, why not? 
(c) Can you develop a receiver that can demodulate the message without knowing the offset frequency f.? 


Problem 4.28: Signal Scrambling 

An excited inventor announces the discovery of a way of using analog technology to render music unlistenable 
without knowing the secret recovery method. The idea is to modulate the bandlimited message m (t) by a 
special periodic signal s (¢) that is zero during half of its period, which renders the message unlistenable and 
superficially, at least, unrecoverable (Figure 4.34). 
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Figure 4.34 


(a) What is the Fourier series for the periodic signal? 

(b) What are the restrictions on the period T so that the message signal can be recovered from m (t) s (t)? 

(c) ELEC 241 students think they have “broken” the inventor’s scheme and are going to announce it to 
the world. How would they recover the original message without having detailed knowledge of the 
modulating signal? 


139 


Solutions to Exercises in Chapter 4 


Solution to Exercise 4.1 (p. 98) 
Because of Euler’s relation, 


A, ues Tt 2 
sin (Qa ft) = —ets27ft _ — eSoft (4.47) 
2) 2) 
1 1 : 
Thus, c, = —, c_1 = —x~-, and the other coefficients are zero. 


2) 2) 
Solution to Exercise 4.2 (p. 102) 


= a This quantity clearly corresponds to the periodic pulse signal’s average value. 


Solution to Exercise 4.3 (p. 103) 
Write the coefficients of the Fourier series in Cartesian form as cy, = A, + 7B, and substitute into the 
expression for the Fourier series. 


, .2akt 
So ae TF = >> (Ant jB) eT 


k=—oo k=—o0o 


Simplifying each term in the sum using Euler’s formula, 
poet Qnkt Inkt 
Csr ie 2 (age gRS (cos (=) Moree (=")) 


Qrkt . Qrkt ; F Qrkt Qrkt 
= A; cos (7) — B,sin (7) +3 (4 sin (7) + Br cos ()) 


We now combine terms that have the same frequency index in magnitude. Because the signal is real-valued, 
the coefficients of the Fourier series have conjugate symmetry: c_~ = cge* or A_, = Ax and B_y = —Bx. 


After we add the positive-indexed and negative-indexed terms, each term in the Fourier series becomes 
2akt Qakt 

2 Ax cos (=) — 2B; sin (ae). To obtain the classic Fourier series (4.11), we must have 2A; = a, and 

2By = —bp. 

Solution to Exercise 4.4 (p. 104) 

The average of a set of numbers is the sum divided by the number of terms. Viewing signal integration as 

the limit of a Riemann sum, the integral corresponds to the average. 

Solution to Exercise 4.5 (p. 104) 


We found that the Fourier series coefficients are given by cy = ae The coefficients are pure imaginary, 
which means ax, = 0. The coefficients of the sine terms are given by b, = —2Im [cx] so that 
4 
—, k odd 
by = wk’ 
0, k even 


Thus, the Fourier series for the square wave is 


4. Inkt 
sa(t)= So —sin— (4.48) 
kE{1,3,...} 


Solution to Exercise 4.6 (p. 106) 
The rms value of a sinusoid equals its amplitude divided by V2. As a half-wave rectified sine wave is zero 
during half of the period, its rms value is A/2 since the integral of the squared half-wave rectified sine wave 
equals half that of a squared sinusoid. 
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Solution to Exercise 4.7 (p. 107) 
Co 


250 lel” 


k=2 


= ; 
2S — lexl? 
harmonic distortion calculations. The factor of two in the numerator and denominator arises because we 
need to sum over negative as well as positive frequency. Because for real-valued signals, c_, = cj, the sums 
over these frequency regions are the same. Clearly, the factors of two cancel. Total harmonic distortion 
is most easily computed in the frequency domain. However, the numerator can be computed in the time 
domain by noting that it equals equals the square of the signal’s rms value (after subtracting the signal’s 
average value) minus the power in the first harmonic. 

Solution to Exercise 4.8 (p. 108) 

Total harmonic distortion in the square wave is 1 — ae = 20%. 

Solution to Exercise 4.9 (p. 111) 


Total harmonic distortion equals The signal’s average value, represented by co, is ignored in 


log. N 
N signals directly encoded require a bandwidth of N/T. Using a binary representation, we need 082 ; 


For N = 128, the binary-encoding scheme has a factor of ae = 0.05 smaller bandwidth. Clearly, binary 


encoding is superior. 

Solution to Exercise 4.10 (p. 111) 

We can use N different amplitude values at only one frequency to represent the various letters. 

Solution to Exercise 4.11 (p. 113) 

Because the filter’s gain at zero frequency equals one, the average output values equal the respective average 
input values. 

Solution to Exercise 4.12 (p. 115) 


Fisin= [sie tar= [sierra =s(-1 


Solution to Exercise 4.13 (p. 116) 
F(F(F(F (s(t) = s(t). We know that F($(f)) = [2,8 (fe taf = [RS (fyetPtOaf = 
s(—t). Therefore, two Fourier transforms applied to s(t) yields s(—t). We need two more to get us back 
where we started. 
Solution to Exercise 4.14 (p. 118) 

The signal is the inverse Fourier transform of the triangularly shaped spectrum, and equals s(t) = 


Ww ( sin (7Wt) ) $ 

tWt 
Solution to Exercise 4.15 (p. 118) 
The result is most easily found in the spectrum’s formula: the power in the signal-related part of x (¢) is half 
the power of the signal s (t). 
Solution to Exercise 4.16 (p. 120) 
The inverse transform of the frequency response is woe! RC u(t). Multiplying the frequency response 
by 1 — e~/?7f4 means subtract from the original signal its time-delayed version. Delaying the frequency 
response’s time-domain version by A results in woe (M/ RC u(t — A). Subtracting from the undelayed 
signal yields pae~'/®° u(t) — gage %)/F° u(t — A). Now we integrate this sum. Because the integral of 
a sum equals the sum of the component integrals (integration is linear), we can consider each separately. 
Because integration and signal-delay are linear, the integral of a delayed signal equals the delayed version of 
the integral. The integral is provided in the example (4.44). 
Solution to Exercise 4.17 (p. 122) 
If the glottis were linear, a constant input (a zero-frequency sinusoid) should yield a constant output. The 
periodic output indicates nonlinear behavior. 
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Solution to Exercise 4.18 (p. 124) 

In the bottom-left panel, the period is about 0.009 s, which equals a frequency of 111 Hz. The bottom-right 
panel has a period of about 0.0065 s, a frequency of 154 Hz. 

Solution to Exercise 4.19 (p. 124) 

Because males have a lower pitch frequency, the spacing between spectral lines is smaller. This closer spacing 
more accurately reveals the formant structure. Doubling the pitch frequency to 300 Hz for Figure 4.16 would 
amount to removing every other spectral line. 
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Chapter 5 


Digital Signal Processing 


5.1 Introduction to Digital Signal Processing’ 


Not only do we have analog signals — signals that are real- or complex-valued functions of a continuous 
variable such as time or space — we can define digital ones as well. Digital signals are sequences, functions 
defined only for the integers. We thus use the notation s(n) to denote a discrete-time one-dimensional signal 
such as a digital music recording and s (m,n) for a discrete-“time” two-dimensional signal like a photo taken 
with a digital camera. Sequences are fundamentally different than continuous-time signals. For example, 
continuity has no meaning for sequences. 

Despite such fundamental differences, the theory underlying digital signal processing mirrors that for ana- 
log signals: Fourier transforms, linear filtering, and linear systems parallel what previous chapters described. 
These similarities make it easy to understand the definitions and why we need them, but the similarities 
should not be construed as “analog wannabes.” We will discover that digital signal processing is not an 
approximation to analog processing. We must explicitly worry about the fidelity of converting analog signals 
into digital ones. The music stored on CDs, the speech sent over digital cellular telephones, and the video 
carried by digital television all evidence that analog signals can be accurately converted to digital ones and 
back again. 

The key reason why digital signal processing systems have a technological advantage today is the com- 
puter: computations, like the Fourier transform, can be performed quickly enough to be calculated as the 
signal is produced,” and programmability means that the signal processing system can be easily changed. 
This flexibility has obvious appeal, and has been widely accepted in the marketplace. Programmability 
means that we can perform signal processing operations impossible with analog systems (circuits). We will 
also discover that digital systems enjoy an algorithmic advantage that contributes to rapid processing 
speeds: Computations can be restructured in non-obvious ways to speed the processing. This flexibility 
comes at a price, a consequence of how computers work. How do computers perform signal processing? 


5.2 Introduction to Computer Organization’ 
5.2.1 Computer Architecture 


To understand digital signal processing systems, we must understand a little about how computers compute. 
The modern definition of a computer is an electronic device that performs calculations on data, presenting 
the results to humans or other computers in a variety of (hopefully useful) ways. 


1This content is available online at http: //cnx.org/content/m10781/2.3/. 

?Taking a systems viewpoint for the moment, a system that produces its output as rapidly as the input arises is said to 
be a real-time system. All analog systems operate in real time; digital ones that depend on a computer to perform system 
computations may or may not work in real time. Clearly, we need real-time signal processing systems. Only recently have 
computers become fast enough to meet real-time requirements while performing non-trivial signal processing. 

3This content is available online at http: //cnx.org/content/m10263/2.28/. 
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1/0 
Interface 


Figure 5.1: Generic computer hardware organization. 


The generic computer contains input devices (keyboard, mouse, A/D (analog-to-digital) converter, etc.), 
a computational unit, and output devices (monitors, printers, D/A converters). The computational unit 
is the computer’s heart, and usually consists of a central processing unit (CPU), a memory, and an 
input/output (I/O) interface. What I/O devices might be present on a given computer vary greatly. 


e A simple computer operates fundamentally in discrete time. Computers are clocked devices, 
in which computational steps occur periodically according to ticks of a clock. This description belies 
clock speed: When you say “I have a 1 GHz computer,” you mean that your computer takes 1 nanosec- 
ond to perform each step. That is incredibly fast! A “step” does not, unfortunately, necessarily mean 
a computation like an addition; computers break such computations down into several stages, which 
means that the clock speed need not express the computational speed. Computational speed is ex- 
pressed in units of millions of instructions/second (Mips). Your 1 GHz computer (clock speed) may 
have a computational speed of 200 Mips. 

e Computers perform integer (discrete-valued) computations. Computer calculations can be 
numeric (obeying the laws of arithmetic), logical (obeying the laws of an algebra), or symbolic (obeying 
any law you like).4 Each computer instruction that performs an elementary numeric calculation — 
an addition, a multiplication, or a division — does so only for integers. The sum or product of two 
integers is also an integer, but the quotient of two integers is likely to not be an integer. How does 
a computer deal with numbers that have digits to the right of the decimal point? This problem is 
addressed by using the so-called floating-point representation of real numbers. At its heart, however, 
this representation relies on integer-valued computations. 


5.2.2 Representing Numbers 


Focusing on numbers, all numbers can represented by the positional notation system.® The b-ary po- 
sitional representation system uses the position of digits ranging from 0 to b-1 to denote a number. The 
quantity b is known as the base of the number system. Mathematically, positional systems represent the 
positive integer n as 


n=) d,b* , d, € {0,...,b-1} (5.1) 
k=0 
and we succinctly express n in base-b as ny = dydn_ 1...do. The number 25 in base-10 equals 2 x 10'+5x 10°, 


so that the digits representing this number are dy = 5, d, = 2, and all other d, equal zero. This same 
number in binary (base-2) equals 11001 (1 x 24+ 1x 2?+0 x 27+0 x 2!+1 x 2°) and 19 in hexadecimal 


4 An example of a symbolic computation is sorting a list of names. 

5 Alternative number representation systems exist. For example, we could use stick figure counting or Roman numerals. 
These were useful in ancient times, but very limiting when it comes to arithmetic calculations: ever tried to divide two Roman 
numerals? 
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(base-16). Fractions between zero and one are represented the same way. 


1: 
f= >> d&b® , d&€ {0,...,b-1} (5.2) 


k=—0o 


All numbers can be represented by their sign, integer and fractional parts. Complex numbers can be thought 
of as two real numbers that obey special rules to manipulate them. 

Humans use base-10, commonly assumed to be due to us having ten fingers. Digital computers use the 
base-2 or binary number representation, each digit of which is known as a bit (binary digit). 


d7| dg| d5| d4| d3| da} dy | dg 
unsigned 8-bit integer 


s | dg] d5| d4| d3| do) dy | do 
signed 8-bit integer 


Ss Ss 


exponent mantissa 
floating point 


Figure 5.2: The various ways numbers are represented in binary are illustrated. The number of bytes 
for the exponent and mantissa components of floating point numbers varies. 


Here, each bit is represented as a voltage that is either “high” or “low,” thereby representing “1” or “0,” 
respectively. To represent signed values, we tack on a special bit—the sign bit—to express the sign. The 
computer’s memory consists of an ordered sequence of bytes, a collection of eight bits. A byte can therefore 
represent an unsigned number ranging from 0 to 255. If we take one of the bits and make it the sign bit, we 
can make the same byte to represent numbers ranging from —128 to 127. But a computer cannot represent 
all possible real numbers. The fault is not with the binary number system; rather having only a finite number 
of bytes is the problem. While a gigabyte of memory may seem to be a lot, it takes an infinite number of bits 
to represent 7. Since we want to store many numbers in a computer’s memory, we are restricted to those 
that have a finite binary representation. Large integers can be represented by an ordered sequence of bytes. 
Common lengths, usually expressed in terms of the number of bits, are 16, 32, and 64. Thus, an unsigned 
32-bit number can represent integers ranging between 0 and 2°? — 1 (4,294,967,295), a number almost big 
enough to enumerate every human in the world!® 


Exercise 5.1 (Solution on p. 191.) 
For both 32-bit and 64-bit integer representations, what are the largest numbers that can be rep- 
resented if a sign bit must also be included. 


While this system represents integers well, how about numbers having nonzero digits to the right of the 
decimal point? In other words, how are numbers that have fractional parts represented? For such numbers, 
the binary representation system is used, but with a little more complexity. The floating-point system 
uses a number of bytes—typically 4 or 8—to represent the number, but with one byte (sometimes two 
bytes) reserved to represent the exponent e of a power-of-two multiplier for the number—the mantissa 
m—expressed by the remaining bytes. 

x= m2° (5.3) 


The mantissa is usually taken to be a binary fraction having a magnitude in the range [5; 1), which means 


that the binary representation is such that d_; = 1.’ The number zero is an exception to this rule, and it is 


6You need one more bit. to do that. 
7In some computers, this normalization is taken to an extreme: the leading binary digit is not explicitly expressed, providing 
an extra bit to represent the mantissa a little more accurately. This convention is known as the hidden-ones notation. 
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the only floating point number having a zero fraction. The sign of the mantissa represents the sign of the 
number and the exponent can be a signed integer. 

A computer’s representation of integers is either perfect or only approximate, the latter situation occurring 
when the integer exceeds the range of numbers that a limited set of bytes can represent. Floating point 
representations have similar representation problems: if the number x can be multiplied/divided by enough 
powers of two to yield a fraction lying between 1/2 and 1 that has a finite binary-fraction representation, the 
number is represented exactly in floating point. Otherwise, we can only represent the number approximately, 
not catastrophically in error as with integers. For example, the number 2.5 equals 0.625 x 27, the fractional 
part of which has an exact binary representation.® However, the number 2.6 does not have an exact binary 
representation, and only be represented approximately in floating point. In single precision floating point 
numbers, which require 32 bits (one byte for the exponent and the remaining 24 bits for the mantissa), the 
number “2.6” will be represented as 2.600000079.... Note that this approximation has a much longer decimal 
expansion. This level of accuracy may not suffice in numerical calculations. Double precision floating 
point numbers consume 8 bytes, and quadruple precision 16 bytes. The more bits used in the mantissa, 
the greater the accuracy. This increasing accuracy means that more numbers can be represented exactly, but 
there are always some that cannot. Such inexact numbers have an infinite binary representation.’ Realizing 
that real numbers can be only represented approximately is quite important, and underlies the entire field 
of numerical analysis, which seeks to predict the numerical accuracy of any computation. 


Exercise 5.2 (Solution on p. 191.) 
What are the largest and smallest numbers that can be represented in 32-bit floating point? in 
64-bit floating point that has sixteen bits allocated to the exponent? Note that both exponent and 
mantissa require a sign bit. 
So long as the integers aren’t too large, they can be represented exactly in a computer using the binary 
positional notation. Electronic circuits that make up the physical computer can add and subtract integers 
without error. (This statement isn’t quite true; when does addition cause problems?) 


5.2.3 Computer Arithmetic and Logic 


The binary addition and multiplication tables are 


0+0=0 O0x0=0 
O0+1=1 0x1=0 
1+1=10 1I1xl=1 
1+0=1 1x0=0 


(5.4) 


Note that if carries are ignored,!° subtraction of two single-digit binary numbers yields the same bit as 
addition. Computers use high and low voltage values to express a bit, and an array of such voltages express 
numbers akin to positional notation. Logic circuits perform arithmetic operations. 


Exercise 5.3 (Solution on p. 191.) 
Add twenty-five and seven in base-2. Note the carries that might occur. Why is the result “nice?” 


The variables of logic indicate truth or falsehood. Af) B, the AND of A and B, represents a statement 
that both A and B must be true for the statement to be true. You use this kind of statement to tell search 
engines that you want to restrict hits to cases where both of the events A and B occur. AUB, the OR of 
A and B, yields a value of truth if either is true. Note that if we represent truth by a “1” and falsehood 
by a “0,” binary multiplication corresponds to AND and addition (ignoring carries) to XOR. 
XOR, the exclusive or operator, equals the union of ALJ B and A(|B. The Irish mathematician George 


8See if you can find this representation. 

°Note that there will always be numbers that have an infinite representation in any chosen positional system. The choice 
of base defines which do and which don’t. If you were thinking that base-10 numbers would solve this inaccuracy, note that 
1/3 = 0.333333.... has an infinite representation in decimal (and binary for that matter), but has finite representation in base-3. 

10A carry means that a computation performed at a given position affects other positions as well. Here, 1+ 1 = 10 is an 
example of a computation that involves a carry. 
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Boole!! discovered this equivalence in the mid-nineteenth century. It laid the foundation for what we now 
call Boolean algebra, which expresses as equations logical statements. More importantly, any computer 
using base-2 representations and arithmetic can also easily evaluate logical statements. This fact makes an 
integer-based computational device much more powerful than might be apparent. 


5.3 The Sampling Theorem” 
5.3.1 Analog-to-Digital Conversion 


Because of the way computers are organized, signal must be represented by a finite number of bytes. This 
restriction means that both the time axis and the amplitude axis must be quantized: They must each be 
a multiple of the integers.!? Quite surprisingly, the Sampling Theorem allows us to quantize the time axis 
without error for some signals. The signals that can be sampled without introducing error are interesting, 
and as described in the next section, we can make a signal “samplable” by filtering. In contrast, no one 
has found a way of performing the amplitude quantization step without introducing an unrecoverable error. 
Thus, a signal’s value can no longer be any real number. Signals processed by digital computers must 
be discrete-valued: their values must be proportional to the integers. Consequently, analog-to-digital 
conversion introduces error. 


5.3.2 The Sampling Theorem 


Digital transmission of information and digital signal processing all require signals to first be “acquired” by 
a computer. One of the most amazing and useful results in electrical engineering is that signals can be 
converted from a function of time into a sequence of numbers without error: We can convert the numbers 
back into the signal with (theoretically) no error. Harold Nyquist, a Bell Laboratories engineer, first derived 
this result, known as the Sampling Theorem, in the 1920s. It found no real application back then. Claude 
Shannon, also at Bell Laboratories, revived the result once computers were made public after World War II. 

The sampled version of the analog signal s(t) is s(n7,), with T, known as the sampling interval. 
Clearly, the value of the original signal at the sampling times is preserved; the issue is how the signal 
values between the samples can be reconstructed since they are lost in the sampling process. To char- 
acterize sampling, we approximate it as the product x(t) = s(t) Pr, (t), with Pr, (t) being the periodic 
pulse signal. The resulting signal, as shown in Figure 5.3, has nonzero values only during the time intervals 
(nT; _ 4, nT, + 4), n€{...,—1,0,1,...}. For our purposes here, we center the periodic pulse signal about 
the origin so that its Fourier series coefficients are real (the signal is even). 


pr, (t)= S> cee (5.5) 
k=—0o 
where 
sin (244) 
a (5.6) 


If the properties of s(t) and the periodic pulse signal are chosen properly, we can recover s(t) from x (t) by 
filtering. 

To understand how signal values between the samples can be “filled” in, we need to calculate the sampled 
signal’s spectrum. Using the Fourier series representation of the periodic sampling signal, 


x(t) = S- che Ts s(t) (5.7) 


llnttp: //www-groups.dcs.st-and.ac.uk/~history/Biographies/Boole. html 
12This content is available online at http: //cnx.org/content/m0050/2.19/. 
13We assume that we do not use floating-point A/D converters. 
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Figure 5.3: The waveform of an example signal is shown in the top plot and its sampled version in the 
bottom. 


Considering each term in the sum separately, we need to know the spectrum of the product of the complex 
exponential and the signal. Evaluating this transform directly is quite easy. 


ae jankt : oa —jan mols 
/ s(t)e 7 evra = f es C *)u=s(s-#) (5.8) 


—oo s 


Thus, the spectrum of the sampled signal consists of weighted (by the coefficients c,) and delayed versions 
of the signal’s spectrum (Figure 5.4). 


co 


X= SD as (s = =) 5.9) 


k=—0o 


In general, the terms in this sum overlap each other in the frequency domain, rendering recovery of the 
original signal impossible. This unpleasant phenomenon is known as aliasing. If, however, we satisfy two 
conditions: 


e The signal s(t) is bandlimited—has power in a restricted frequency range—to W Hz, and 
e the sampling interval 7), is small enough so that the individual components in the sum do not overlap— 
T; < 1/2W, 


aliasing will not occur. In this delightful case, we can recover the original signal by lowpass filtering x (t) 
with a filter having a cutoff frequency equal to W Hz. These two conditions ensure the ability to recover a 
bandlimited signal from its sampled version: We thus have the Sampling Theorem. 


Exercise 5.4 (Solution on p. 191.) 
The Sampling Theorem (as stated) does not mention the pulse width A. What is the effect of this 
parameter on our ability to recover a signal from its samples (assuming the Sampling Theorem’s 
two conditions are met)? 


1 
The frequency Te known today as the Nyquist frequency and the Shannon sampling frequency, 


corresponds to the highest frequency at which a signal can contain energy and remain compatible with 
the Sampling Theorem. High-quality sampling systems ensure that no aliasing occurs by unceremoniously 
lowpass filtering the signal (cutoff frequency being slightly lower than the Nyquist frequency) before sampling. 
Such systems therefore vary the anti-aliasing filter’s cutoff frequency as the sampling rate varies. Because 
such quality features cost money, many sound cards do not have anti-aliasing filters or, for that matter, 
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Figure 5.4: The spectrum of some bandlimited (to W Hz) signal is shown in the top plot. If the 
sampling interval T’ is chosen too large relative to the bandwidth W, aliasing will occur. In the bottom 
plot, the sampling interval is chosen sufficiently small to avoid aliasing. Note that if the signal were not 
bandlimited, the component spectra would always overlap. 


post-sampling filters. They sample at high frequencies, 44.1 kHz for example, and hope the signal contains 
no frequencies above the Nyquist frequency (22.05 kHz in our example). If, however, the signal contains 
frequencies beyond the sound card’s Nyquist frequency, the resulting aliasing can be impossible to remove. 


Exercise 5.5 (Solution on p. 191.) 
To gain a better appreciation of aliasing, sketch the spectrum of a sampled square wave. For 


simplicity consider only the spectral repetitions centered at TP 0, qT Let the sampling interval 


T, be 1; consider two values for the square wave’s period: 3.5 and 4, Note in particular where the 
spectral lines go as the period decreases; some will move to the left and some to the right. What 
property characterizes the ones going the same direction? 

If we satisfy the Sampling Theorem’s conditions, the signal will change only slightly during each pulse. As 
we narrow the pulse, making A smaller and smaller, the nonzero values of the signal s (t) pr, (t) will simply 
be s(nT,), the signal’s samples. If indeed the Nyquist frequency equals the signal’s highest frequency, at 
least two samples will occur within the period of the signal’s highest frequency sinusoid. In these ways, the 
sampling signal captures the sampled signal’s temporal variations in a way that leaves all the original signal’s 
structure intact. 

Exercise 5.6 (Solution on p. 191.) 
What is the simplest bandlimited signal? Using this signal, convince yourself that less than two 


samples /period will not suffice to specify it. If the sampling rate Tr is not high enough, what signal 
s 


would your resulting under-sampled signal become? 
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5.4 Amplitude Quantization” 


The Sampling Theorem says that if we sample a bandlimited signal s (t) fast enough, it can be recovered with- 
out error from its samples s(nT,), n € {...,—1,0,1,...}. Sampling is only the first phase of acquiring data 
into a computer: Computational processing further requires that the samples be quantized: analog values 
are converted into digital form. In short, we will have performed analog-to-digital (A/D) conversion. 


signal amplitude-quantized 
and sampled signal 


-1 


Figure 5.5: A three-bit A/D converter assigns voltage in the range [—1,1] to one of eight integers 
between 0 and 7. For example, all inputs having values lying between 0.5 and 0.75 are assigned the 
integer value six and, upon conversion back to an analog value, they all become 0.625. The width of a 


2 
single quantization interval A equals =. The bottom panel shows a signal going through the analog- 


to-digital converter, where B is the number of bits used in the A/D conversion process (3 in the case 
depicted here). First it is sampled, then amplitude-quantized to three bits. Note how the sampled signal 
waveform becomes distorted after amplitude quantization. For example the two signal values between 0.5 


and 0.75 become 0.625. This distortion is irreversible; it can be reduced (but not eliminated) by using 
more bits in the A/D converter. 


A phenomenon reminiscent of the errors incurred in representing numbers on a computer prevents signal 
amplitudes from being converted with no error into a binary number representation. In analog-to-digital 
conversion, the signal is assumed to lie within a predefined range. Assuming we can scale the signal without 
affecting the information it expresses, we’ll define this range to be [—1, 1]. Furthermore, the A/D converter 
assigns amplitude values in this range to a set of integers. A B-bit converter produces one of the integers 
{0, 1,...,27 — 1} for each sampled input. Figure 5.5 shows how a three-bit A/D converter assigns input 
values to the integers. We define a quantization interval to be the range of values assigned to the same 
integer. Thus, for our example three-bit A/D converter, the quantization interval A is 0.25; in general, it is 

2 


9B . 
Exercise 5.7 


(Solution on p. 191.) 
Recalling the plot of average daily highs in this frequency domain problem (Problem 4.5), why is 


this plot so jagged? Interpret this effect in terms of analog-to-digital conversion. 


Because values lying anywhere within a quantization interval are assigned the same value for computer 
processing, the original amplitude value cannot be recovered without error. Typically, the D/A 


14This content is available online at http: //cnx.org/content/m0051/2.22/. 
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converter, the device that converts integers to amplitudes, assigns an amplitude equal to the value lying 
halfway in the quantization interval. The integer 6 would be assigned to the amplitude 0.625 in this scheme. 
The error introduced by converting a signal from analog to digital form by sampling and amplitude quanti- 
zation then back again would be half the quantization interval for each amplitude value. Thus, the so-called 
A/D error equals half the width of a quantization interval: 5H: As we have fixed the input-amplitude range, 
the more bits available in the A/D converter, the smaller the quantization error. 

To analyze the amplitude quantization error more deeply, we need to compute the signal-to-noise ratio, 
which equals the ratio of the signal power and the quantization error power. Assuming the signal is a 


sinusoid, the signal power is the square of the rms amplitude: power (s) = (+) = 5. The illustration 


(Figure 5.6) details a single quantization interval. Its width is A and the quantization error is denoted by 


Figure 5.6: A single quantization interval is shown, along with a typical signal’s value before amplitude 
quantization s(nT;) and after Q (s(nT)). € denotes the error thus incurred. 


e. To find the power in the quantization error, we note that no matter into which quantization interval the 
signal’s value falls, the error will have the same characteristics. To calculate the rms value, we must square 
the error and average it over the interval. 


rms (€) = x/ e? de 
Pao (5.10) 
A2 2 
2 


2 
Since the quantization interval width for a B-bit converter equals 3B = 2-(8-1) | we find that the signal-to- 


noise ratio for the analog-to-digital conversion process equals 


1 
5 3 

SNR = = - ao = 6B + 10log,) 1.5 dB (5.11) 
12 


Thus, every bit increase in the A/D converter yields a 6 dB increase in the signal-to-noise ratio. The constant 
term 10 log; 1.5 equals 1.76. 
Exercise 5.8 (Solution on p. 191.) 
This derivation assumed the signal’s amplitude lay in the range [—1, 1]. What would the amplitude 
quantization signal-to-noise ratio be if it lay in the range [—A, A]? 
Exercise 5.9 (Solution on p. 191.) 
How many bits would be required in the A/D converter to ensure that the maximum amplitude 
quantization error was less than 60 db smaller than the signal’s peak value? 


Exercise 5.10 (Solution on p. 191.) 
Music on a CD is stored to 16-bit accuracy. To what signal-to-noise ratio does this correspond? 
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Once we have acquired signals with an A/D converter, we can process them using digital hardware or software. 
It can be shown that if the computer processing is linear, the result of sampling, computer processing, and 
un-sampling is equivalent to some analog linear system. Why go to all the bother if the same function can 
be accomplished using analog techniques? Knowing when digital processing excels and when it does not is 
an important issue. 


5.5 Discrete-Time Signals and Systems” 


Mathematically, analog signals are functions having as their independent variables continuous quantities, 
such as space and time. Discrete-time signals are functions defined on the integers; they are sequences. As 
with analog signals, we seek ways of decomposing discrete-time signals into simpler components. Because 
this approach leads to a better understanding of signal structure, we can exploit that structure to represent 
information (create ways of representing information with signals) and to extract information (retrieve the 
information thus represented). For symbolic-valued signals, the approach is different: We develop a common 
representation of all symbolic-valued signals so that we can embody the information they contain in a 
unified way. From an information representation perspective, the most important issue becomes, for both 
real-valued and symbolic-valued signals, efficiency: what is the most parsimonious and compact way to 
represent information so that it can be extracted later. 


5.5.1 Real- and Complex-valued Signals 


A discrete-time signal is represented symbolically as s(n), where n = {...,—1,0,1,...}. 


Sn 
1 


aie 


Figure 5.7: The discrete-time cosine signal is plotted as a stem plot. Can you find the formula for this 
signal? 


We usually draw discrete-time signals as stem plots to emphasize the fact they are functions defined only on 
the integers. We can delay a discrete-time signal by an integer just as with analog ones. A signal delayed 
by m samples has the expression s(n — m). 


5.5.2 Complex Exponentials 


The most important signal is, of course, the complex exponential sequence. 
s(n) = ef?nfn (5.12) 


Note that the frequency variable f is dimensionless and that adding an integer to the frequency of the 
discrete-time complex exponential has no effect on the signal’s value. 


el2n(f+m)n = e)2rfnejanmn 
(5.13) 


= ei2nfn 


This derivation follows because the complex exponential evaluated at an integer multiple of 27 equals one. 
Thus, we need only consider frequency to have a value in some unit-length interval. 


15This content is available online at http: //cnx.org/content/m10342/2.15/. 
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5.5.3 Sinusoids 


Discrete-time sinusoids have the obvious form s(n) = Acos(2rfn+¢). As opposed to analog complex 
exponentials and sinusoids that can have their frequencies be any real value, frequencies of their discrete- 
time counterparts yield unique waveforms only when f lies in the interval (-5, 3]: This choice of frequency 
interval is arbitrary; we can also choose the frequency to lie in the interval [0, 1). How to choose a unit-length 
interval for a sinusoid’s frequency will become evident later. 


5.5.4 Unit Sample 


The second-most important discrete-time signal is the unit sample, which is defined to be 


= {1 as (5.14) 


0 otherwise 


6n 


O19 Or Nn 


Figure 5.8: The unit sample. 


Examination of a discrete-time signal’s plot, like that of the cosine signal shown in Figure 5.7, reveals that 
all signals consist of a sequence of delayed and scaled unit samples. Because the value of a sequence at 
each integer m is denoted by s(m) and the unit sample delayed to occur at m is written 6 (n — m), we can 
decompose any signal as a sum of unit samples delayed to the appropriate location and scaled by the signal 
value. 


s(n)= S > s(m)d(n—m) (5.15) 


m>=—Cco 


This kind of decomposition is unique to discrete-time signals, and will prove useful subsequently. 
5.5.5 Unit Step 


The unit step in discrete-time is well-defined at the origin, as opposed to the situation with analog signals. 


1 n>0 
u(n) = i‘ er (5.16) 


5.5.6 Symbolic Signals 


An interesting aspect of discrete-time signals is that their values do not need to be real numbers. We do 
have real-valued discrete-time signals like the sinusoid, but we also have signals that denote the sequence of 
characters typed on the keyboard. Such characters certainly aren’t real numbers, and as a collection of pos- 
sible signal values, they have little mathematical structure other than that they are members of a set. More 
formally, each element of the symbolic-valued signal s(n) takes on one of the values {a1,...,aK} which 
comprise the alphabet A. This technical terminology does not mean we restrict symbols to being mem- 
bers of the English or Greek alphabet. They could represent keyboard characters, bytes (8-bit quantities), 
integers that convey daily temperature. Whether controlled by software or not, discrete-time systems are 
ultimately constructed from digital circuits, which consist entirely of analog circuit elements. Furthermore, 
the transmission and reception of discrete-time signals, like e-mail, is accomplished with analog signals and 
systems. Understanding how discrete-time and analog signals and systems intertwine is perhaps the main 
goal of this course. 
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5.5.7 Discrete-Time Systems 


Discrete-time systems can act on discrete-time signals in ways similar to those found in analog signals and 
systems. Because of the role of software in discrete-time systems, many more different systems can be 
envisioned and “constructed” with programs than can be with analog signals. In fact, a special class of 
analog signals can be converted into discrete-time signals, processed with software, and converted back into 
an analog signal, all without the incursion of error. For such signals, systems can be easily produced in 
software, with equivalent analog realizations difficult, if not impossible, to design. 


5.6 Discrete-Time Fourier Transform (DTFT)” 
The Fourier transform of the discrete-time signal s(n) is defined to be 


S(t l= S- s(n) e92nfn (5.17) 


n=— Co 


Frequency here has no units. As should be expected, this definition is linear, with the transform of a 
sum of signals equaling the sum of their transforms. Real-valued signals have conjugate-symmetric spectra: 
S (e-3?7F) = s* (eff). 

Exercise 5.11 (Solution on p. 192.) 

A special property of the discrete-time Fourier transform is that it is periodic with period one: 

Ss (e727(F+1) =§ (ef?*F). Derive this property from the definition of the DTFT. 


Because of this periodicity, we need only plot the spectrum over one period to understand completely the 
spectrum’s structure; typically, we plot the spectrum over the frequency range [-3, 5]. When the signal 
is real-valued, we can further simplify our plotting chores by showing the spectrum only over (0, 5]: the 
spectrum at negative frequencies can be derived from positive-frequency spectral values. 

When we obtain the discrete-time signal via sampling an analog signal, the Nyquist frequency (p. 148) 


corresponds to the discrete-time frequency 4. To show this, note that a sinusoid having a frequency equal 
Y 9 g y 


1 
to the Nyquist frequency oT has a sampled waveform that equals 


Ss 


1 n 
cos (2rspnTs) = cos (mn) = (-1) 


.27n . 
The exponential in the DTFT at frequency 5 equals e / 2. =e -I7™" = (—1)”, meaning that discrete-time 


frequency equals analog frequency multiplied by the sampling interval 


fo = faTs (5.18) 


fp and f, represent discrete-time and analog frequency variables, respectively. Figure 5.4 inspires another 
way of deriving this result. As the duration of each pulse in the periodic sampling signal pr, (t) narrows, the 
amplitudes of the signal’s spectral repetitions, which are governed by the Fourier series coefficients (4.10) of 
pr, (t), become increasingly equal. Examination of the periodic pulse signal (Figure 4.1) reveals that as A 


decreases, the value of co, the largest Fourier coefficient, decreases to zero: |co| = . Thus, to maintain 


s 
a mathematically viable Sampling Theorem, the amplitude A must increase as A’ becoming infinitely large 


as the pulse duration decreases. Practical systems use a small value of A, say 0.1 -7, and use amplifiers to 


rescale the signal. Thus, the sampled signal’s spectrum becomes periodic with period tT Thus, the Nyquist 
s 


1 1 
frequency oT corresponds to the frequency 5° 


Ss 


16This content is available online at http: //cnx.org/content/m10247/2.31/. 
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Example 5.1 
Let’s compute the discrete-time Fourier transform of the exponentially decaying sequence s(n) = 
a” u(n), where u(n) is the unit-step sequence. Simply plugging the signal’s expression into the 


Fourier transform formula, 
co 


S (e9?*F) = se a” u(n)e 227 Fn 
oes (5.19) 
= ys (ae~J?7F)" 
n=0 
This sum is a special case of the geometric series. 
co A 1 
n=0 
Let a = ae~/?"J. As long as |a| < 1, |a| < 1. We find our Fourier transform to be 
S(ee) = a la| <1 (5.21) 
1 — ae7327F? 
Using Euler’s relation, we can express the magnitude and phase of this spectrum. 
|S (e2?"F) | = (5.22) 


va — acos (27f))? + a2sin? (27f) 


Ct asin (27f) ) 
ZS (ce?!) = —tan era enon (5.23) 
No matter what value of a we choose, the above formulae clearly demonstrate the periodic nature 
of the spectra of discrete-time signals. Figure 5.9 shows indeed that the spectrum is a periodic 
function. We need only consider the spectrum between —4 and 4 to unambiguously define it. 
When a > 0, we have a lowpass spectrum—the spectrum diminishes as frequency increases from 0 
to +—with increasing a leading to a greater low frequency content; for a < 0, we have a highpass 


2 
spectrum (Figure 5.10). 


Example 5.2 
Analogous to the analog pulse signal, let’s find the spectrum of the length-N pulse sequence. 


1, f0<n<N-1 
s(n) = tov (5.24) 
0, otherwise 
The Fourier transform of this sequence has the form of a truncated geometric series. 
N-1 
Ss (er) = s genie (5.25) 
n=0 
For the so-called finite geometric series, we know that 
N+no-1 N 
l-a 
n= Qn 5.26 
SE eae ars (5.26) 
n=no 


for all values of a. 

Exercise 5.12 (Solution on p. 192.) 
Derive this formula for the finite geometric series sum. The “trick” is to consider the difference 
between the series’ sum and the sum of the series multiplied by a. 
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-45 | 


Figure 5.9: The spectrum of the exponential signal (a = 0.5) is shown over the frequency range [—2, 2], 
clearly demonstrating the periodicity of all discrete-time spectra. The angle has units of degrees. 


Spectral Magnitude (dB) 


Angle (degrees) 


Figure 5.10: The spectra of several exponential signals are shown. What is the apparent relationship 
between the spectra for a = 0.5 and a = —0.5? 


Applying this result yields (Figure 5.11) 
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sin (Nx) 

sin (x) ’ 
function dsinc (x). Thus, our transform can be concisely expressed as S$ (e/?"f) = e~I™fN—Vdsine (mf). 
The discrete-time pulse’s spectrum contains many ripples, the number of which increase with N, the pulse’s 
duration. 


The ratio of sine functions has the generic form of which is known as the discrete-time sinc 
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Figure 5.11: The spectrum of a length-ten pulse is shown. Can you explain the rather complicated 
appearance of the phase? 


The inverse discrete-time Fourier transform is easily derived from the following relationship: 


3 . 
[ e i201 fm ojanfn df -_ fs ifm=n 


= 0, ifmA#n 
=d(m—n) (5.28) 
Therefore, we find that 
1 
a S(e sf) elt in dt — S> s( —j2nrfm el?t in ar 
=} -4 Sn 
: (5.29) 
= e J2nf(m—n) 
= > af a" : df 
m 2 
= s(n) 
The Fourier transform pairs in discrete-time are 
S (one) = y s(n) g anim 
a (5.30) 


s(n) = fs (e"*) clan fn df 
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The properties of the discrete-time Fourier transform mirror those of the analog Fourier transform. The 
DTFT properties table!” shows similarities and differences. One important common property is Parseval’s 


Theorem. 
co a 


S> |s(n)[? =f. 15 (ce?) [Pag (5.31) 


i 
n=— oo 2 


To show this important property, we simply substitute the Fourier transform expression into the frequency- 
domain expression for power. 


ie 


i 
2 


IS (e? 4) Pag = / : 


—i 
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(= s(n) ee) (= s* iyeretn) df 


n 


; (5.32) 
= SS s(n) s* (n) / EleT in n) ap 


n,m =o 
Using the orthogonality relation (5.28), the integral equals 6(m—n), where 6(n) is the unit sample (Fig- 
ure 5.8). Thus, the double sum collapses into a single sum because nonzero values occur only when n = m, 
giving Parseval’s Theorem as a result. We term )>,, s* (n) the energy in the discrete-time signal s (n) in spite 
of the fact that discrete-time signals don’t consume (or produce for that matter) energy. This terminology 
is a carry-over from the analog world. 


Exercise 5.13 (Solution on p. 192.) 
Suppose we obtained our discrete-time signal from values of the product s(t) pr, (t), where the 
duration of the component pulses in pr, (t) is A. How is the discrete-time signal energy related to 
the total energy contained in s(t)? Assume the signal is bandlimited and that the sampling rate 
was chosen appropriate to the Sampling Theorem’s conditions. 


The following table (5.1) summarizes the properties of the discrete-time Fourier transform. Note the 
similarities and differences with the corresponding properties of the Fourier transform of analog signals 
(page 117). The notable differences are the scaling properties that, for analog signals, are expressed as s(at). 
For discrete-time signals, the scaling factor a must be an integer and the resulting spectrum must be periodic 
with period 1. For example, the spectrum of x(n) = s(n/2), n even, x(n) = 0, n odd, is derived as follows. 


S- s(n/2)eJ27 Fr — > s(m)eF27F-2™ In = 2m] 


n even m 


-g (ci?) 


Exercise 5.14 (Solution on p. 192) 
Note that the spectrum of this stretched-signal example has a period equal to 1/2 and that the 
spectrum is a “compressed” version of s(n)’s spectrum. Suppose that this stretched signal were 
passed through a lowpass filter having a cutoff frequency of 1/4. What effect would this filtering 
have on the stretched signal? How would you interpret this effect? 


5.7 Discrete Fourier Transforms (DFT)” 


The discrete-time Fourier transform (and the continuous-time transform as well) can be evaluated when we 
have an analytic expression for the signal. Suppose we just have a signal, such as the speech signal used in the 
previous chapter, for which there is no formula. How then would you compute the spectrum? For example, 
how did we compute a spectrogram such as the one shown in the speech signal example (Figure 4.17)? The 
Discrete Fourier Transform allows the computation of spectra from discrete-time data. While in discrete- 
time we can exactly calculate spectra, for analog signals no similar exact spectrum computation exists. 
For analog-signal spectra, use must build special devices, which turn out in most cases to consist of A/D 


1™Tiscrete-Time Fourier Transform Properties” http<://cnx.org/content/m0506/latest/ 
18This content is available online at http: //cnx.org/content/m10249/2.27/. 
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Index Domain 


Frequency Domain 


Linearity a181(n) + agso(n) a1; (e)?"F) + azSp (3?) 
Periodicity (period 1) | all s(n) Ss (c?27f) —S§ (ef2" f+) 
Conjugate Symmetry | s(n) ER S (ci27) = S* (e~#?7F) 
Even Symmetry s(n) = s(—n) S (e27f) = S (e-9?7f) 
Odd Symmetry s(n) = —s(—n) S (ef7F) = —§ (e-927F) 
M =...,—-M,0,M,2M,... : 
Signal Stretch x(n) ESS : »0,.M, 2M, xX (cif) = 9g (ci2"Mf) 
0 otherwise 
M—1 
Downsampling s(nM) S- Ss (er) 
m=0 
Delay s(n — no) e~ Pring (ei2tF) 
Complex Modulation | e/?"fo" s(n) S (e2*(f-fo)) 
5 (ei2"F—Io)) 4. § (ei2mF +o) 
Cosine Modulation cos(27 fon)s(n) (e ) : (¢ ) 
5 (ei2"F—fo)) — § (elf +o) 
Sine Modulation sin(27 fon)s(n) G oy te 
J 
Multiplication by n ns(n) acne (e727) 
jan df 
Sum >. s(n) g (ci2*0) 
1/2 oar 
Value at Origin (0) S (eS) df 
-1/2 
1/2 . 
Parseval’s Theorem 3, 8? (n) / |S (e??"F)|° of 
-1/2 
Table 5.1 


converters and discrete-time computations. Certainly discrete-time spectral analysis is more flexible than 
continuous-time spectral analysis. 
The formula for the DTFT (5.17) is a sum, which conceptually can be easily computed save for two 


issues. 


e Signal duration. The sum extends over the signal’s duration, which must be finite to compute the 
signal’s spectrum. It is exceedingly difficult to store an infinite-length signal in any case, so we’ll 
assume that the signal extends over [0, N — 1]. 

e Continuous frequency. Subtler than the signal duration issue is the fact that the frequency variable 
is continuous: It may only need to span one period, like [-$ 4] or [0,1], but the DTFT formula as it 


2° 2 


stands requires evaluating the spectra at all frequencies within a period. Let’s compute the spectrum 


at a few frequencies; the most obvious ones are the equally spaced ones f = KR ke {0,...,K —1}. 
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We thus define the discrete Fourier transform(DFT) to be 


N-1 
S(b)= So s(n)eP e, REG ank HA} (5.33) 
n=0 


ok 
Here, S$ (k) is shorthand for S'{ e/?"* ). 


By choosing the value of K, we can compute the spectrum at as many equally spaced frequencies as 
we like. Note that you can think about this computationally motivated choice as sampling the spectrum; 
more about this interpretation later. The issue now is how many frequencies are enough to capture how 
the spectrum changes with frequency. One way of answering this question is determining an inverse discrete 
Fourier transform formula: given S'(k), k = {0,...,& —1} how do we find s(n), n = {0,...,N—1}? 


K-1 
.2ank 
Presumably, the formula will be of the form s(n) = S~ S(k)e?” & . Substituting the DFT formula in this 
k=0 


prototype inverse transform yields 


kK-1 /N-1 .2nmk .2nnk 
s(n) = (Sse K one ee 


Note that the orthogonality relation we use so often has a different character now. 


K-1 . 
.2rkm .2nxkn = = le 

Soe He AP ANE ifm ={n, (n+ K),(n42K),...} 

k=0 


4 (5.35) 
0, otherwise 


We obtain nonzero value whenever the two indices differ by multiples of K. We can express this result as 
K 5°) (6(m—n-—Ik)). Thus, our formula becomes 


N-1 lee) 
s(n)= S> s(m)K S* 5(m—n-1k) (5.36) 
m=0 l=—oo 
The integers n and m both range over {0,..., N — 1}. To have an inverse transform, we need the sum to be a 


single unit sample for m, n in this range. If it did not, then s(n) would equal a sum of values, and we would 
not have a valid transform: Once going into the frequency domain, we could not get back unambiguously! 
Clearly, the term / = 0 always provides a unit sample (we’ll take care of the factor of K soon). If we evaluate 
the spectrum at fewer frequencies than the signal’s duration, the term corresponding tom = n+ K will 
also appear for some values of m, n = {0,..., N —1}. This situation means that our prototype transform 
equals s(n) + s(n + K) for some values of n. The only way to eliminate this problem is to require K > N: 
We must have at least as many frequency samples as the signal’s duration. In this way, we can return from 
the frequency domain we entered via the DFT. 


Exercise 5.15 (Solution on p. 192.) 
When we have fewer frequency samples than the signal’s duration, some discrete-time signal values 
equal the sum of the original signal values. Given the sampling interpretation of the spectrum, 
characterize this effect a different way. 
Another way to understand this requirement is to use the theory of linear equations. If we write out the 
expression for the DFT as a set of linear equations, 


s(0)+s(1)+---+s(N-1)=S(0) 


2m , 21) 
s(0)+s(lje 7K +---+8(N—-l)e? K =S(1) 
(5.37) 
_2n(K—1) _2n(N—1)(K-1) 


s(0)+s(l)e% K +---+8(N-l)ed  K =S(K-1) 
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we have K equations in N unknowns if we want to find the signal from its sampled spectrum. This require- 
ment is impossible to fulfill if K < N; we must have kK > N. Our orthogonality relation essentially says that 
if we have a sufficient number of equations (frequency samples), the resulting set of equations can indeed be 
solved. 

By convention, the number of DFT frequency values Kk is chosen to equal the signal’s duration N. The 
discrete Fourier transform pair consists of 


Discrete Fourier Transform Pair 


rete 2nnk 
S(k) = s(nje? N 
n=0 
7 (5.38) 
a 1 S(k j PERE 
s(n) = W » e€ 


Since the DFT is merely a sampled version of the DTFT, the table shown on page 159 applies to the DFT 
as well, but with one notable exception: signals having even or odd symmetry. Because signals amenable to 
the DFT are defined only for n = 0..., N —1, what does s(—n) mean? Because the result of computing the 
DFT, then the IDFT of the result, is a periodic version of the original signal, we consider the original signal 
to be periodic with period N. In this case, an even signal would satisfy s(—n) = s(N — n). Computing 
the DFT of an even or odd signal would yield the result s(n) = +s(N —n) <— 45(N—k). Also note 
that using the table for real-valued signals, we would conclude that S(k) = S*(—k). However, the DFT’s 
result is only defined for positive k. Note that the DFT spectrum is periodic with period N, implying that 
S(—k) = S(N —k). Therefore, the DFT of all real-valued signals must have the property S(k) = S*(N—k). 


Exercise 5.16 (Solution on p. ??) 
What does this conjugate symmetry property say about S(N/2) when N, the transform length, is 
even? 


5.8 DFT: Computational Complexity” 


We now have a way of computing the spectrum for an arbitrary signal: The Discrete Fourier Transform 
(DFT) (5.33) computes the spectrum at N equally spaced frequencies from a length- N sequence. An issue 
that never arises in analog “computation,” like that performed by a circuit, is how much work it takes to 
perform the signal processing operation such as filtering. In computation, this consideration translates to 
the number of basic computational steps required to perform the needed processing. The number of steps, 
known as the complexity, becomes equivalent to how long the computation takes (how long must we wait 
for an answer). Complexity is not so much tied to specific computers or programming languages but to how 
many steps are required on any computer. Thus, a procedure’s stated complexity says that the time taken 
will be proportional to some function of the amount of data used in the computation and the amount 
demanded. 

For example, consider the formula for the discrete Fourier transform. For each frequency we choose, we 
must multiply each signal value by a complex number and add together the results. For a real-valued signal, 
each real-times-complex multiplication requires two real multiplications, meaning we have 2N multiplications 
to perform. To add the results together, we must keep the real and imaginary parts separate. Adding N 
numbers requires N — 1 additions. Consequently, each frequency requires 2N + 2(N —1) = 4N — 2 basic 
computational steps. As we have N frequencies, the total number of computations is N (4N — 2). 

In complexity calculations, we only worry about what happens as the data lengths increase, and take the 
dominant term—here the 4N? term—as reflecting how much work is involved in making the computation. 
As multiplicative constants don’t matter since we are making a “proportional to” evaluation, we find the 
DFT is an O (N 2) computational procedure. This notation is read “order N-squared.” Thus, if we double 
the length of the data, we would expect that the computation time to approximately quadruple. 


19This content is available online at http: //cnx.org/content/m0503/2.11/. 
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Exercise 5.17 (Solution on p. 192.) 
In making the complexity evaluation for the DFT, we assumed the data to be real. Three questions 
emerge. First of all, the spectra of such signals have conjugate symmetry, meaning that negative 


N 
frequency components (k = {3 +1,...,N+ i} in the DFT (5.33)) can be computed from the 


corresponding positive frequency components. Does this symmetry change the DFT’s complexity? 
Secondly, suppose the data are complex-valued; what is the DFT’s complexity now? Finally, a less 
important but interesting question is suppose we want K frequency values instead of N; now what 
is the complexity? 


5.9 Fast Fourier Transform (FFT)” 


One wonders if the DFT can be computed faster: Does another computational procedure — an algorithm— 
exist that can compute the same quantity, but more efficiently. We could seek methods that reduce the 
constant of proportionality, but do not change the DFT’s complexity O (N ae Here, we have something 
more dramatic in mind: Can the computations be restructured so that a smaller complexity results? 

In 1965, IBM researcher James Cooley?! and Princeton faculty member John Tukey?? developed what is 
now known as the Fast Fourier Transform (FFT). It is an algorithm for computing that DFT that has order 
O(NlogN) for certain length inputs. Now when the length of data doubles, the spectral computational 
time will not quadruple as with the DFT algorithm; instead, it approximately doubles. Later research showed 
that no algorithm for computing the DFT could have a smaller complexity than the FFT. Surprisingly, 
historical work has shown that Gauss? in the early nineteenth century developed the same algorithm, but 
did not publish it! After the FFT’s rediscovery, not only was the computation of a signal’s spectrum greatly 
speeded, but also the added feature of algorithm meant that computations had flexibility not available to 
analog implementations. 


Exercise 5.18 (Solution on p. 192.) 
Before developing the FFT, let’s try to appreciate the algorithm’s impact. Suppose a short-length 
transform takes 1 ms. We want to calculate a transform of a signal that is 10 times longer. Compare 
how much longer a straightforward implementation of the DFT would take in comparison to an 
FFT, both of which compute exactly the same quantity. 


To derive the FFT, we assume that the signal’s duration is a power of two: N = 2". Consider what happens 
to the even-numbered and odd-numbered elements of the sequence in the DFT calculation. 


2m 2k get N= 2) 
S(k) =s(0)+s(2)e7 N +---+8(N-2)e7 
Qnk 2n(Q+1)k 2n(N=2+1)k 
Let W Sua 6 Gee oe 
Ink __.2n(N/2—1)k 
=|s(0)+s(2)e “N24.---+s(N-2)e’ NP | (5.39) 
peor __.2n(N/2—1)k see 
+ |s(1)+s(3)e NZ 4+..-4+8(N-l)e’? XP vas 


Each term in square brackets has the form of a * length DFT. The first one is a DFT of the even-numbered 
elements, and the second of the odd-numbered elements. The first DFT is combined with the second 


27k 
multiplied by the complex exponential e-7 N_. The half-length transforms are each evaluated at frequency 
indices k = 0, ..., N—1. Normally, the number of frequency indices in a DFT calculation range between zero 
and the transform length minus one. The computational advantage of the FFT comes from recognizing 


20This content is available online at http: //cnx.org/content/m10250/2.17/. 
*lnttps: //en.wikipedia.org/wiki/James_Cooley 
22nttps://en.wikipedia.org/wiki/John_Tukey 
23nttp://www-groups.dcs.st-and.ac.uk/~history/Biographies/Gauss .html 
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Figure 5.12: The initial decomposition of a length-8 DFT into the terms using even- and odd-indexed 
inputs marks the first phase of developing the FFT algorithm. When these half-length transforms are 
successively decomposed, we are left with the diagram shown in the bottom panel that depicts the 
length-8 FFT computation. 


the periodic nature of the discrete Fourier transform. The FFT simply reuses the computations made in 
27k 
the half-length transforms and combines them through additions and the multiplication by e~? N, which is 


N 
not periodic over 3 to rewrite the length-N DFT. Figure 5.12 illustrates this decomposition. As it stands, 


N 
we now compute two length-— transforms (complexity 20 (N a 4)), multiply one of them by the complex 


exponential (complexity O (N)), and add the results (complexity O(N)). At this point, the total complexity 
is still dominated by the half-length DFT calculations, but the proportionality coefficient has been reduced. 

Now for the fun. Because N = 2”, each of the half-length transforms can be reduced to two quarter- 
length transforms, each of these to two eighth-length ones, etc. This decomposition continues until we are 
left with length-2 transforms. This transform is quite simple, involving only additions. Thus, the first stage 


N 
of the FFT has length-2 transforms (see the bottom part of Figure 5.12). Pairs of these transforms are 


combined by adding one to the other multiplied by a complex exponential. Each pair requires 6 additions 


5N 
and 4 multiplications, giving a total number of computations equaling 10 - [> a This number of 


computations does not change from stage to stage. Because the number of stages, the number of times the 
length can be divided by two, equals log,N, the complexity of the FFT is O (Nlog,N). 

Doing an example will make computational savings more obvious. Let’s look at the details of a length-8 
DFT. As shown on Figure 5.13, we first decompose the DFT into two length-4 DFTs, with the outputs added 
and subtracted together in pairs. Considering Figure 5.13 as the frequency index goes from 0 through 7, 
we recycle values from the length-4 DFTs into the final calculation because of the periodicity of the DFT 
output. Examining how pairs of outputs are collected together, we create the basic computational element 
known as a butterfly(Figure 5.13). 
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Figure 5.13: The basic computational element of the fast Fourier transform is the butterfly. It takes 
two complex numbers, represented by a and b, and forms the quantities shown. Each butterfly requires 
one complex multiplication and two complex additions. 


By considering together the computations involving common output frequencies from the two half-length 
DFTs, we see that the two complex multiplies are related to each other, and we can reduce our computational 
work even further. By further decomposing the length-4 DFTs into two length-2 DFTs and combining their 
outputs, we arrive at the diagram summarizing the length-8 fast Fourier transform (Figure 5.12). Although 
most of the complex multiplies are quite simple (multiplying by e~J” means negating real and imaginary 


parts), let’s count those for purposes of evaluating the complexity as full complex multiplies. We have — = 4 


complex multiplies and N = 8 additions for each stage and log,N = 3 stages, making the number of basic 
3N 
computations a 08a as predicted. 


Exercise 5.19 (Solution on p. 192.) 
Note that the ordering of the input sequence in the two parts of Figure 5.12 aren’t quite the same. 
Why not? How is the ordering determined? 


Other “fast” algorithms were discovered, all of which make use of how many common factors the transform 
length N has. In number theory, the number of prime factors a given integer has measures how composite 
it is. The numbers 16 and 81 are highly composite (equaling 2+ and 3* respectively), the number 18 is less so 
(2! - 37), and 17 not at all (it’s prime). In over thirty years of Fourier transform algorithm development, the 
original Cooley-Tukey algorithm is far and away the most frequently used. It is so computationally efficient 
that power-of-two transform lengths are frequently used regardless of what the actual length of the data. 


Exercise 5.20 (Solution on p. 192.) 
Suppose the length of the signal were 500? How would you compute the spectrum of this signal 
using the Cooley-Tukey algorithm? What would the length N of the transform be? 


5.10 Spectrograms”™ 


We know how to acquire analog signals for digital processing by A/D conversion (Section 5.4), consisting 
of pre-filtering, sampling and amplitude quantization, and to compute spectra of discrete-time signals using 
the FFT algorithm (Section 5.9). Let’s put these various components together to learn how the spectrogram 
shown in Figure 5.14, which is used to analyze speech, is calculated. The speech was sampled at a rate of 
11.025 kHz and passed through a 16-bit A/D converter. 


POINT OF INTEREST: Music compact discs (CDs) encode their signals at a sampling rate of 
44.1 kHz. We’ll learn the rationale for this number later. The 11.025 kHz sampling rate for the 
speech is 1/4 of the CD sampling rate, and was the lowest available sampling rate commensurate 
with speech signal bandwidths available on my computer. 


Exercise 5.21 (Solution on p. 192.) 
Looking at Figure 5.14 the signal lasted a little over 1.2 seconds. How long was the sampled signal 
(in terms of samples)? What was the datarate during the sampling process in bps (bits per second)? 


24This content is available online at http: //cnx.org/content/m0505/2.19/. 
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Figure 5.14 


Assuming the computer storage is organized in terms of bytes (8-bit quantities), how many bytes 
of computer memory does the speech consume? 


The resulting discrete-time signal, shown in the bottom of Figure 5.14, clearly changes its character with 
time. To display these spectral changes, the long signal was sectioned into frames: comparatively short, 
contiguous groups of samples. Conceptually, a Fourier transform of each frame is calculated using the FFT. 
Each frame is not so long that significant signal variations are retained within a frame, but not so short 
that we lose the signal’s spectral character. Roughly speaking, the speech signal’s spectrum is evaluated 
over successive time segments and stacked side by side so that the z-axis corresponds to time and the y-axis 
frequency, with color indicating the spectral amplitude. 

An important detail emerges when we examine each framed signal (Figure 5.15). At the frame’s edges, 
the signal may change very abruptly, a feature not present in the original signal. A transform of such a 
segment reveals a curious oscillation in the spectrum, an artifact directly related to this sharp amplitude 
change. A better way to frame signals for spectrograms is to apply a window: Shape the signal values 
within a frame so that the signal decays gracefully as it nears the edges. This shaping is accomplished 
by multiplying the framed signal by the sequence w(n). In sectioning the signal, we essentially applied a 
rectangular window: w(n) =1,0<n< N-—1. A much more graceful window is the Hanning window; 
it has the cosine shape w(n) = $(1—cos(2m7n/N)). As shown in Figure 5.15, this shaping greatly reduces 
spurious oscillations in each frame’s spectrum. Considering the spectrum of the Hanning windowed frame, 
we find that the oscillations resulting from applying the rectangular window obscured a formant (the one 
located at a little more than half the Nyquist frequency). 
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Figure 5.15: The top waveform is a segment 1024 samples long taken from the beginning of the “Rice 
University” phrase. Computing Figure 5.14 involved creating frames, here demarcated by the vertical 
lines, that were 256 samples long and finding the spectrum of each. If a rectangular window is applied 
(corresponding to extracting a frame from the signal), oscillations appear in the spectrum (middle of 
bottom row). Applying a Hanning window gracefully tapers the signal toward frame edges, thereby 
yielding a more accurate computation of the signal’s spectrum at that moment of time. 


Figure 5.16: In comparison with the original speech segment shown in the upper plot, the non- 
overlapped Hanning windowed version shown below it is very ragged. Clearly, spectral information 
extracted from the bottom plot could well miss important features present in the original. 


Exercise 5.22 (Solution on p. 193.) 
What might be the source of these oscillations? To gain some insight, what is the length- 2N 
discrete Fourier transform of a length-N pulse? The pulse emulates the rectangular window, and 
certainly has edges. Compare your answer with the length-2N transform of a length-N Hanning 
window. 
If you examine the windowed signal sections in sequence to examine windowing’s effect on signal amplitude, 
we see that we have managed to amplitude-modulate the signal with the periodically repeated window 
(Figure 5.16). To alleviate this problem, frames are overlapped (typically by half a frame duration). This 
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Figure 5.17: The original speech segment and the sequence of overlapping Hanning windows applied 
to it are shown in the upper portion. Frames were 256 samples long and a Hanning window was applied 
with a half-frame overlap. A length-512 FFT of each frame was computed, with the magnitude of the 
first 257 FFT values displayed vertically, with spectral amplitude values color-coded. 


solution requires more Fourier transform calculations than needed by rectangular windowing, but the spectra 
are much better behaved and spectral changes are much better captured. 
The speech signal, such as shown in Figure 5.14, is sectioned into overlapping, equal-length frames, with 

a Hanning window applied to each frame. The spectra of each of these is calculated, and displayed in 
spectrograms with frequency extending vertically, window time location running horizontally, and spectral 
magnitude color-coded. Figure 5.17 illustrates these computations. 

Exercise 5.23 (Solution on p. 193.) 

Why the specific values of 256 for N and 512 for K? Another issue is how was the length-512 

transform of each length-256 windowed frame computed? 


5.11 Discrete-Time Systems” 


When we developed analog systems, interconnecting the circuit elements provided a natural starting place for 
constructing useful devices. In discrete-time signal processing, we are not limited by hardware considerations 
but by what can be constructed in software. 


Exercise 5.24 (Solution on p. 193.) 
One of the first analog systems we described was the amplifier (Section 2.6.2). We found that 
implementing an amplifier was difficult in analog systems, requiring an op-amp at least. What is 
the discrete-time implementation of an amplifier? Is this especially hard or easy? 


In fact, we will discover that frequency-domain implementation of systems, wherein we multiply the input 
signal’s Fourier transform by a frequency response, is not only a viable alternative, but also a computationally 
efficient one. We begin with discussing the underlying mathematical structure of linear, shift-invariant 
systems, and devise how software filters can be constructed. 


25This content is available online at http: //cnx.org/content/m0507/2.5/. 
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5.12 Discrete-Time Systems in the Time-Domain” 


A discrete-time signal s(n) is delayed by no samples when we write s(n — 79), with ng > 0. Choosing ng 
to be negative advances the signal along the integers. As opposed to analog delays, discrete-time delays can 
only be integer valued. In the frequency domain, delaying a signal corresponds to a linear phase shift of the 
signal’s discrete-time Fourier transform: s(n — no) > e-J?7/"0 (€)?7/), 

Linear discrete-time systems have the superposition property. 


S [a1 (n) + agx2 (n)| = a1S [x1 (n)] + a2S [xo (n)] (5.40) 


A discrete-time system is called shift-invariant (analogous to time-invariant analog systems (p. 24)) if 
delaying the input delays the corresponding output. If S [x (n)] = y(n), then a shift-invariant system has 
the property 

S [x(n — no)] = y(n — no) (5.41) 


We use the term shift-invariant to emphasize that delays can only have integer values in discrete-time, while 
in analog signals, delays can be arbitrarily valued. 

We want to concentrate on systems that are both linear and shift-invariant. It will be these that allow us 
the full power of frequency-domain analysis and implementations. Because we have no physical constraints 
in “constructing” such systems, we need only a mathematical specification. In analog systems, the differen- 
tial equation specifies the input-output relationship in the time-domain. The corresponding discrete-time 
specification is the difference equation. 


y(n) = ay(n—1) +--+ + apy (n— p) + box (n) + bia (n — 1) +--+ + dgx (n — q) (5.42) 


Here, the output signal y (n) is related to its past values y (n — 1), 1 = {1,...,p}, and to the current and past 
values of the input signal x(n). The system’s characteristics are determined by the choices for the number 
of coefficients p and q and the coefficients’ values {a1,...,ap} and {bo, b1,..., bg}. 


ASIDE: There is an asymmetry in the coefficients: where is ag? This coefficient would multiply the 
y(n) term in (5.42). We have essentially divided the equation by it, which does not change the 
input-output relationship. We have thus created the convention that ao is always one. 


As opposed to differential equations, which only provide an implicit description of a system (we must 
somehow solve the differential equation), difference equations provide an explicit way of computing the 
output for any input. We simply express the difference equation by a program that calculates each output 
from the previous output values, and the current and previous inputs. 

Difference equations are usually expressed in software with for loops. A MATLAB program that would 
compute the first 1000 values of the output has the form 


for n=1:1000 
y(n) = sum(a.*y(n-1:-1:n-p)) + sum(b.*x(n:-1:n-q)); 
end 


An important detail emerges when we consider making this program work; in fact, as written it has (at 
least) two bugs. What input and output values enter into the computation of y(1)? We need values for 
y (0) ,y(—1),... values we have not yet computed. To compute them, we would need more previous values of 
the output, which we have not yet computed. To compute these values, we would need even earlier values, ad 
infinitum. The way out of this predicament is to specify the system’s initial conditions: we must provide 
the p output values that occurred before the input started. These values can be arbitrary, but the choice 
does impact how the system responds to a given input. One choice gives rise to a linear system: Make the 
initial conditions zero. The reason lies in the definition of a linear system: The only way that the output to 


26This content is available online at http: //cnx.org/content/m10251/2.23/. 


169 


a sum of signals can be the sum of the individual outputs occurs when the initial conditions in each case are 
zero. 


Exercise 5.25 (Solution on p. 193.) 
The initial condition issue resolves making sense of the difference equation for inputs that start at 
some index. However, the program will not work because of a programming, not conceptual, error. 
What is it? How can it be “fixed?” 


Example 5.3 
Let’s consider the simple system having p = 1 and q = 0. 


y(n) = ay (n — 1) + ba (n) (5.43) 


To compute the output at some index, this difference equation says we need to know what the 
previous output y(n — 1) and what the input signal is at that moment of time. In more detail, let’s 
compute this system’s output to a unit-sample input: x(n) = 6(n). Because the input is zero for 
negative indices, we start by trying to compute the output at n = 0. 


y (0) = ay(-1) +6 (5.44) 


What is the value of y (—1)? Because we have used an input that is zero for all negative indices, it 
is reasonable to assume that the output is also zero. Certainly, the difference equation would not 
describe a linear system if the input that is zero for all time did not produce a zero output. With 
this assumption, y(—1) = 0, leaving y(0) = b. For n > 0, the input unit-sample is zero, which 
leaves us with the difference equation y(n) = ay(n—1), n > 0. We can envision how the filter 
responds to this input by making a table. 


y(n) = ay (n — 1) + bd (n) (5.45) 
n | x(n) | y(n) 
—1 0 0) 
0 1 
1 0 ba 
2 0 ba? 
n 0 ba” 
Table 5.2 


Coefficient values determine how the output behaves. The parameter 6 can be any value, and 
serves as a gain. The effect of the parameter a is more complicated (Table 5.2). If it equals zero, 
the output simply equals the input times the gain b. For all non-zero values of a, the output 
lasts forever; such systems are said to be IIR (Infinite Impulse Response). The reason for this 
terminology is that the unit sample also known as the impulse (especially in analog situations), and 
the system’s response to the “impulse” lasts forever. If a is positive and less than one, the output 
is a decaying exponential. When a = 1, the output is a unit step. If a is negative and greater 
than —1, the output oscillates while decaying exponentially. When a = —1, the output changes 
sign forever, alternating between b and —b. More dramatic effects when |a| > 1; whether positive 
or negative, the output signal becomes larger and larger, growing exponentially. 

Positive values of a are used in population models to describe how population size increases 
over time. Here, n might correspond to generation. The difference equation says that the number 
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Figure 5.18: The input to the simple example system, a unit sample, is shown at the top, with the 
outputs for several system parameter values shown below. 
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Figure 5.19: The plot shows the unit-sample response of a length-5 boxcar filter. 


in the next generation is some multiple of the previous one. If this multiple is less than one, the 
population becomes extinct; if greater than one, the population flourishes. The same difference 
equation also describes the effect of compound interest on deposits. Here, n indexes the times at 
which compounding occurs (daily, monthly, etc.), a equals the compound interest rate plus one, 
and b = 1 (the bank provides no gain). In signal processing applications, we typically require that 
the output remain bounded for any input. For our example, that means that we restrict |a| < 1 
and choose values for it and the gain according to the application. 


Exercise 5.26 (Solution on p. 193.) 
Note that the difference equation (5.42), 


y(n) = ary (n—1) +--+ + apy (nm — p) + box (mn) + bra (n — 1) + +++ + bgx (n — @) 


does not involve terms like y(n +1) or (n+ 1) on the equation’s right side. Can such terms also 
be included? Why or why not? 


Example 5.4 
A somewhat different system has no “a” coefficients. Consider the difference equation 
1 
y(n) = [e(n) t+ +e(n—at)) (5.46) 
Because this system’s output depends only on current and previous input values, we need not 
1 
be concerned with initial conditions. When the input is a unit-sample, the output equals — for 


n = {0,...,q—1}, then equals zero thereafter. Such systems are said to be FIR (Finite Impulse 
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Response) because their unit sample responses have finite duration. Plotting this response (Fig- 
ure 5.19) shows that the unit-sample response is a pulse of width g and height —. This waveform 


is also known as a boxcar, hence the name boxcar filter given to this system. We’ll derive its 
frequency response and develop its filtering interpretation in the next section. For now, note that 
the difference equation says that each output value equals the average of the input’s current and 
previous values. Thus, the output equals the running average of input’s previous q values. Such a 
system could be used to produce the average weekly temperature (q = 7) that could be updated 
daily. 


5.13 Discrete-Time Systems in the Frequency Domain” 


As with analog linear systems, we need to find the frequency response of discrete-time systems. We used 
impedances to derive directly from the circuit’s structure the frequency response. The only structure we have 
so far for a discrete-time system is the difference equation. We proceed as when we used impedances: let the 
input be a complex exponential signal. When we have a linear, shift-invariant system, the output should also 
be a complex exponential of the same frequency, changed in amplitude and phase. These amplitude and phase 
changes comprise the frequency response we seek. The complex exponential input signal is x(n) = Xe??7/”. 
Note that this input occurs for all values of n. No need to worry about initial conditions here. Assume the 
output has a similar form: y(n) = Ye/?7/". Plugging these signals into the fundamental difference equation 
(5.42), we have 


Veltrin — g Vel) 4 ot ApY €f?7F(n—P) + byte rt™ 4 hy Xt Dk ee bg XeF?7F(n—9) (5.47) 


The assumed output does indeed satisfy the difference equation if the output complex amplitude is related 
to the input amplitude by 

bp + bye 927F 40H bge I?m ah 
eer es earner ee 


Y 


This relationship corresponds to the system’s frequency response or, by another name, its transfer function. 
We find that any discrete-time system defined by a difference equation has a transfer function given by 


bp + be 927F Hee te bge I2r af 
1— aye J27f a ee Ape J27Ps 


(5.48) 


H (ci?!) = 


In analogy with the results for analog signals, the transfer function should relate the discrete-time Fourier 
transform of the system’s output to the input’s Fourier transform. To show this result, we evaluate the DTFT 
of equation (5.42). Noting that the Fourier transform is linear and that y(n — no) @ e-/?/no . Y (e?"f), 
we obtain 


¥ (e!"f) = [ayeS27F 4... + ape 927/P] V (F275) + [by + bre 927F 4... + bye F27F4] X (ef? Ff) 
Simplification gives use the expected result: 
Y (e*) = H (e??"S) X (eS) (5.49) 


where H (e/?"/) is given by (5.48). 
Example 5.5 
The frequency response of the simple IIR system (difference equation given in a previous example 
(Example 5.3)) is given by 

b 


Ee) =e ay 


(5.50) 


This Fourier transform occurred in a previous example; the exponential signal spectrum (Fig- 
ure 5.10) portrays the magnitude and phase of this transfer function. When the filter coefficient a 


27This content is available online at http: //cnx.org/content/m0510/2.14/. 


172 CHAPTER 5. DIGITAL SIGNAL PROCESSING 


is positive, we have a lowpass filter; negative a results in a highpass filter. The larger the coefficient 
in magnitude, the more pronounced the lowpass or highpass filtering. 


Example 5.6 
The length-g boxcar filter (difference equation found in a previous example (Example 5.4)) has the 
frequency response 


Her freee etna (5.51) 
This expression amounts to the Fourier transform of the boxcar signal (Figure 5.19). There we 
found that this frequency response has a magnitude equal to the absolute value of dsinc (7 f); see 

the length-10 filter’s frequency response (Figure 5.11). We see that boxcar filters-length-q signal 
averagers—have a lowpass behavior, having a cutoff frequency of a 


Exercise 5.27 (Solution on p. 193.) 
Suppose we multiply the boxcar filter’s coefficients by a sinusoid: b,, = F cos (27 fom). Use Fourier 
transform properties to determine the transfer function. How would you characterize this system: 
Does it act like a filter? If so, what kind of filter and how do you control its characteristics with 
the filter’s coefficients? 


These examples illustrate the point that systems described (and implemented) by difference equations serve 
as filters for discrete-time signals. The filter’s order is given by the number p of denominator coefficients 
in the transfer function (if the system is IIR) or by the number gq of numerator coefficients if the filter is 
FIR. When a system’s transfer function has both terms, the system is usually IIR, and its order equals p 
regardless of g. By selecting the coefficients and filter type, filters having virtually any frequency response 
desired can be designed. This design flexibility can’t be found in analog systems. In the next section, we 
detail how analog signals can be filtered by computers, offering a much greater range of filtering possibilities 
than is possible with circuits. 


5.14 Filtering in the Frequency Domain™ 


Because we are interested in actual computations rather than analytic calculations, we must consider the 
details of the discrete Fourier transform. To compute the length-N DFT, we assume that the signal has a 
duration less than or equal to N. Because frequency responses have an explicit frequency-domain specification 
(5.47) in terms of filter coefficients, we don’t have a direct handle on which signal has a Fourier transform 
equaling a given frequency response. Finding this signal is quite easy. First of all, note that the discrete- 
time Fourier transform of a unit sample equals one for all frequencies. Because the input and output of 
linear, shift-invariant systems are related to each other by Y (e/?"/) = H (e/?"S) X (e/?"/), a unit-sample 
input, which has X (c??"F) = 1, results in the output’s Fourier transform equaling the system’s 
transfer function. 

Exercise 5.28 (Solution on p. 193.) 

This statement is a very important result. Derive it yourself. 
In the time-domain, the output for a unit-sample input is known as the system’s unit-sample response, 
and is denoted by h(n). Combining the frequency-domain and time-domain interpretations of a linear, shift- 
invariant system’s unit-sample response, we have that h(n) and the transfer function are Fourier transform 


pairs in terms of the discrete-time Fourier transform. 
h(n) <> H (e??"/) (5.52) 


Returning to the issue of how to use the DFT to perform filtering, we can analytically specify the frequency 
response, and derive the corresponding length-N DFT by sampling the frequency response. 


H(k)=H jee) AO eg} (5.53) 


28This content is available online at http: //cnx.org/content/m10257/2.17/. 
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Computing the inverse DFT yields a length-N signal no matter what the actual duration of the unit- 
sample response might be. If the unit-sample response has a duration less than or equal to N (it’s a FIR 
filter), computing the inverse DFT of the sampled frequency response indeed yields the unit-sample response. 
If, however, the duration exceeds N, errors are encountered. The nature of these errors is easily explained 
by appealing to the Sampling Theorem. By sampling in the frequency domain, we have the potential for 
aliasing in the time domain (sampling in one domain, be it time or frequency, can result in aliasing in the 
other) unless we sample fast enough. Here, the duration of the unit-sample response determines the minimal 
sampling rate that prevents aliasing. For FIR systems — they by definition have finite-duration unit sample 
responses — the number of required DFT samples equals the unit-sample response’s duration: N > q. 


Exercise 5.29 (Solution on p. 193.) 
Derive the minimal DFT length for a length-qg unit-sample response using the Sampling Theorem. 
Because sampling in the frequency domain causes repetitions of the unit-sample response in the 
time domain, sketch the time-domain result for various choices of the DFT length N. 


Exercise 5.30 (Solution on p. 193.) 
Express the unit-sample response of a FIR filter in terms of difference equation coefficients. Note 
that the corresponding question for IIR filters is far more difficult to answer: Consider the example 
(Example 5.5). 


For IIR systems, we cannot use the DFT to find the system’s unit-sample response: aliasing of the unit- 
sample response will always occur. Consequently, we can only implement an IIR filter accurately in the time 
domain with the system’s difference equation. Frequency-domain implementations are restricted to 
FIR filters. 

Another issue arises in frequency-domain filtering that is related to time-domain aliasing, this time when 
we consider the output. Assume we have an input signal having duration N, that we pass through a FIR 
filter having a length-q + 1 unit-sample response. What is the duration of the output signal? The difference 
equation for this filter is 

y(n) = box (n) +--+ + bgx (n— q) (5.54) 


This equation says that the output depends on current and past input values, with the input value g samples 
previous defining the extent of the filter’s memory of past input values. For example, the output at index 
N, depends on x (N,) (which equals zero), x (Nz, — 1), through x (N, — q). Thus, the output returns to zero 
only after the last input value passes through the filter’s memory. As the input signal’s last value occurs at 
index N,, —1, the last nonzero output value occurs when n—q = N,—1 orn =q+N,-—1. Thus, the output 
signal’s duration equals g + N,. 

Exercise 5.31 (Solution on p. 193.) 

In words, we express this result as “The output’s duration equals the input’s duration plus the 

filter’s duration minus one.” Demonstrate the accuracy of this statement. 


The main theme of this result is that a filter’s output extends longer than either its input or its unit-sample 
response. Thus, to avoid aliasing when we use DFTs, the dominant factor is not the duration of input or 
of the unit-sample response, but of the output. Thus, the number of values at which we must evaluate the 
frequency response’s DFT must be at least q+ Ny, and we must compute the same length DFT of the input. 
To accommodate a shorter signal than DFT length, we simply zero-pad the input: Ensure that for indices 
extending beyond the signal’s duration that the signal is zero. Frequency-domain filtering, diagrammed in 
Figure 5.20, is accomplished by storing the filter’s frequency response as the DFT H (k), computing the 
input’s DFT X (k), multiplying them to create the output’s DFT Y (k) = H (k) X (k), and computing the 
inverse DFT of the result to yield y(n). 
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Figure 5.20: To filter a signal in the frequency domain, first compute the DFT of the input, multiply 
the result by the sampled frequency response, and finally compute the inverse DFT of the product. The 
DFT’s length must be at least the sum of the input’s and unit-sample response’s duration minus one. 
We calculate these discrete Fourier transforms using the fast Fourier transform algorithm, of course. 


Before detailing this procedure, let’s clarify why so many new issues arose in trying to develop a frequency- 
domain implementation of linear filtering. The frequency-domain relationship between a filter’s input and 
output is always true: Y (e/?"/) = H (e/?"f) X (e/?*f). The Fourier transforms in this result are discrete- 
time Fourier transforms; for example, X (e!?"/) = >, x(n) e~/?"/". Unfortunately, using this relationship 
to perform filtering is restricted to the situation when we have analytic formulas for the frequency response 
and the input signal. The reason why we had to “invent” the discrete Fourier transform (DFT) has the 
same origin: The spectrum resulting from the discrete-time Fourier transform depends on the continuous 
frequency variable f. That’s fine for analytic calculation, but computationally we would have to make an 
uncountably infinite number of computations. 


NOTE: Did you know that two kinds of infinities can be meaningfully defined? A countably 
infinite quantity means that it can be associated with a limiting process associated with integers. 
An uncountably infinite quantity cannot be so associated. The number of rational numbers is 
countably infinite (the numerator and denominator correspond to locating the rational by row and 
column; the total number so-located can be counted, voila!); the number of irrational numbers is 
uncountably infinite. Guess which is “bigger?” 


The DFT computes the Fourier transform at a finite set of frequencies — samples the true spectrum — 
which can lead to aliasing in the time-domain unless we sample sufficiently fast. The sampling interval here 


il 
is — for a length-K DFT: faster sampling to avoid aliasing thus requires a longer transform calculation. 


Since the longest signal among the input, unit-sample response and output is the output, it is that signal’s 
duration that determines the transform length. We simply extend the other two signals with zeros (zero-pad) 
to compute their DFTs. 


Example 5.7 

Suppose we want to average daily stock prices taken over last year to yield a running weekly 
average (average over five trading sessions). The filter we want is a length-5 averager (as shown in 
Figure 5.19), and the input’s duration is 253 (365 calendar days minus weekend days and holidays). 
The output duration will be 253 + 5 — 1 = 257, and this determines the transform length we need 
to use. Because we want to use the FFT, we are restricted to power-of-two transform lengths. We 
need to choose any FFT length that exceeds the required DFT length. As it turns out, 256 is a 
power of two (2° = 256), and this length just undershoots our required length. To use frequency 
domain techniques, we must use length-512 fast Fourier transforms. 

Figure 5.21 shows the input and the filtered output. The MATLAB programs that compute the 

filtered output in the time and frequency domains are 


Time Domain 
h = [1 111 1]/5; 
y = filter(h,1,[djia zeros(1,4)]); 


Daily Average 
Weekly Average 


Dow-Jones Industrial Avera 


0 50 100 150 200 250 
Trading Day (1997) 


Figure 5.21: The blue line shows the Dow Jones Industrial Average from 1997, and the red one the 
length-5 boxcar-filtered result that provides a running weekly of this market index. Note the “edge” 
effects in the filtered output. 


Frequency Domain 

h = [1 11 1 1)/5; 
DJIA = fft(djia, 512); 
H = fft(h, 512); 

Y = H.*X; 

y = ifft(y); 


NOTE: The filter program has the feature that the length of its output equals the length of its 
input. To force it to produce a signal having the proper length, the program zero-pads the input 
appropriately. 


MATLAB’s fft function automatically zero-pads its input if the specified transform length (its 
second argument) exceeds the signal’s length. The frequency domain result will have a small 
imaginary component — largest value is 2.2 x 10~'! — because of the inherent finite precision 
nature of computer arithmetic. Because of the unfortunate misfit between signal lengths and 
favored FFT lengths, the number of arithmetic operations in the time-domain implementation is 
far less than those required by the frequency domain version: 514 versus 62,271. If the input signal 
had been one sample shorter, the frequency-domain computations would have been more than a 
factor of two less (28,696), but far more than in the time-domain implementation. 

An interesting signal processing aspect of this example is demonstrated at the beginning and 
end of the output. The ramping up and down that occurs can be traced to assuming the input is 
zero before it begins and after it ends. The filter “sees” these initial and final values as the difference 
equation passes over the input. These artifacts can be handled in two ways: we can just ignore the 
edge effects or the data from previous and succeeding years’ last and first week, respectively, can 
be placed at the ends. 
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time domain 


frequency domain 


Number of Computations 


The colored plane plots (Nz + q) (2g + 1), the computations required to implement directly an FIR filter 
with a difference equation. The red plane displays the number of computations needed by the frequency 
domain approach: 6 (Nz + q) +5 (Nz + q) log, (Nz + 4). 


5.15 Efficiency of Frequency-Domain Filtering” 

To determine for what signal and filter durations a time- or frequency-domain implementation would be 
the most efficient, we need only count the computations required by each. For the time-domain, difference- 
equation approach, we need (N,+q) (2q¢ +1). As Figure 5.20 shows, the frequency-domain approach requires 
three Fourier transforms but we need to only compute two of them: typically, H(k) is computed once and 


5K 
stored. Each requires - logs K computations for a length-kK FFT. We must multiply two complex-valued 


length-K spectra; that requires 6K computations. Because the output-signal-duration-determined length 
must be at least N, + q, we must compare 


(Nz +9) (2q +1) + 6 (Nz +9) 4 2-9 (Nz +4) logs (Ne +4) 


Exact analytic evaluation of this comparison is quite difficult (we have a transcendental equation to solve). 
Insight into this comparison is best obtained by dividing by Nz + q. 


2g+1< > 64 5log, (Nz + g) 


With this manipulation, we are evaluating the number of computations per sample. For any given value 
of the filter’s order q, the right side, the number of frequency-domain computations, will exceed the left if 
the signal’s duration is long enough. However, for filter durations greater than about 10, as long as the 
input is at least 10 samples, the frequency-domain approach is faster so long as the FF T’s power-of-two 
constraint is advantageous. Graphically, the parameter range over which the frequency-domain approach 
results in fewer computations is shown in the figure. As the analysis and the plot shows, only for short filter 
durations does the difference equation approach require fewer computations. 

The frequency-domain approach is not yet viable; what will we do when the input signal is infinitely long? 
The difference equation scenario fits perfectly with the envisioned digital filtering structure (Figure 5.24), but 
so far we have required the input to have limited duration (so that we could calculate its Fourier transform). 
The solution to this problem is quite simple: Section the input into frames, filter each, and add the results 
together. To section a signal means expressing it as a linear combination of length-N, non-overlapping 


2°This content is available online at http: //cnx.org/content/m10279/2.16/. 
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Sectioned Input 


Figure 5.22: The noisy input signal is sectioned into length-48 frames, each of which is filtered using 
frequency-domain techniques. Each filtered section is added to other outputs that overlap to create the 
signal equivalent to having filtered the entire input. The sinusoidal component of the signal is shown as 
the red dashed line. 


“chunks.” Because the filter is linear, filtering a sum of terms is equivalent to summing the results of filtering 


each term. 
Co CoO 


x(n) = z(n-—mN,) > y(n) = S- y(n—mN,) (5.55) 
m=—co m=—Cco 
As illustrated in Figure 5.22, note that each filtered section has a duration longer than the input. Conse- 
quently, we must literally add the filtered sections together, not just butt them together. 

Computational considerations reveal a substantial advantage for a frequency-domain implementation 
over a time-domain one. The number of computations for a time-domain implementation essentially remains 
constant whether we section the input or not. Thus, the number of computations for each output is 2q + 1. 
In the frequency-domain approach, computation counting changes because we need only compute the filter’s 
frequency response H (k) once, which amounts to a fixed overhead. We need only compute two DFTs and 
multiply them to filter a section. Letting N, denote a section’s length, the number of computations for 
a section amounts to (N, + q) log, (Ni + q)+6(N,+¢). In addition, we must add the filtered outputs 
together; the number of terms to add corresponds to the excess duration of the output compared with the 


input (q). The frequency-domain approach thus requires log, (N, + q)+64 NV y computations per output 
x vq 


value. For even modest filter orders, the frequency-domain approach is much faster. 
Exercise 5.32 (Solution on p. 193.) 
Show that as the section length increases, the frequency domain approach becomes increasingly 
more efficient. 


Note that the choice of section duration is arbitrary. Once the filter is chosen, we should section so that the 
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Figure 5.23: The figure shows the unit-sample response of a length-17 Hanning filter on the left and 
the frequency response on the right. This filter functions as a lowpass filter having a cutoff frequency of 
about 0.1. 


required FFT length is precisely a power of two: Choose N; so that N, + q = 2¥. 

Implementing the digital filter shown in the A/D block diagram (Figure 5.24) with a frequency-domain 
implementation requires some additional signal management not required by time-domain implementations. 
Conceptually, a real-time, time-domain filter could accept each sample as it becomes available, calculate 
the difference equation, and produce the output value, all in less than the sampling interval T,. Frequency- 
domain approaches don’t operate on a sample-by-sample basis; instead, they operate on sections. They 
filter in real time by producing N, outputs for the same number of inputs faster than N,7. Because they 
generally take longer to produce an output section than the sampling interval duration, we must filter one 
section while accepting into memory the next section to be filtered. In programming, the operation of 
building up sections while computing on previous ones is known as buffering. Buffering can also be used 
in time-domain filters as well but isn’t required. 


Example 5.8 

We want to lowpass filter a signal that contains a sinusoid and a significant amount of noise. The 
example shown in Figure 5.22 shows a portion of the noisy signal’s waveform. If it weren’t for the 
overlaid sinusoid, discerning the sine wave in the signal is virtually impossible. One of the primary 
applications of linear filters is noise removal: preserve the signal by matching filter’s passband 
with the signal’s spectrum and greatly reduce all other frequency components that may be present 
in the noisy signal. 

A smart Rice engineer has selected a FIR filter having a unit-sample response corresponding a 
period-17 sinusoid: h(n) = 7(1 — cos (2n/17)), n = {0,...,16}, which makes g = 16. Its fre- 
quency response (determined by computing the discrete Fourier transform) is shown in Figure 5.23. 
To apply, we can select the length of each section so that the frequency-domain filtering approach 
is maximally efficient: Choose the section length Nz, so that N, + q is a power of two. To use 
a length-64 FFT, each section must be 48 samples long. Filtering with the difference equation 
would require 33 computations per output while the frequency domain requires a little over 16; this 
frequency-domain implementation is over twice as fast! Figure 5.22 shows how frequency-domain 
filtering works. 

We note that the noise has been dramatically reduced, with a sinusoid now clearly visible in the 
filtered output. Some residual noise remains because noise components within the filter’s passband 
appear in the output as well as the signal. 

Exercise 5.33 (Solution on p. 193.) 
Note that when compared to the input signal’s sinusoidal component, the output’s sinusoidal com- 
ponent seems to be delayed. What is the source of this delay? Can it be removed? 
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5.16 Discrete-Time Filtering of Analog Signals” 


Because of the Sampling Theorem, we can process, in particular filter, analog signals “with a computer” by 
constructing the system shown in Figure 5.24. To use this system, we are assuming that the input signal has 
a lowpass spectrum and can be bandlimited without affecting important signal aspects. Bandpass signals 
can also be filtered digitally, but require a more complicated system. Highpass signals cannot be filtered 
digitally. Note that the input and output filters must be analog filters; trying to operate without them can 
lead to potentially very inaccurate digitization. 


ae 
x(t) LPF AVS (Q[x(nTs)]| ——— y(n) LPF y(t) 
> ; Digital —_> 
sm ig Pl Maa aca inci 

1 1 1 
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Figure 5.24: To process an analog signal digitally, the signal x (t) must be filtered with an anti-aliasing 
filter (to ensure a bandlimited signal) before A/D conversion. This lowpass filter (LPF) has a cutoff 
frequency of W Hz, which determines allowable sampling intervals T;. The greater the number of bits 
in the amplitude quantization portion Q |-] of the A/D converter, the greater the accuracy of the entire 
system. The resulting digital signal x(n) can now be filtered in the time-domain with a difference 
equation or in the frequency domain with Fourier transforms. The resulting output y(n) then drives a 
D/A converter and a second anti-aliasing filter (having the same bandwidth as the first one). 


Another implicit assumption is that the digital filter can operate in real time: The computer and the 
filtering algorithm must be sufficiently fast so that outputs are computed faster than input values arrive. 
The sampling interval, which is determined by the analog signal’s bandwidth, thus determines how long our 
program has to compute each output y(n). The computational complexity for calculating each output with 
a difference equation (5.42) is O(p+q). Frequency domain implementation of the filter is also possible. 
The idea begins by computing the Fourier transform of a length-N portion of the input x(n), multiplying 
it by the filter’s transfer function, and computing the inverse transform of the result. This approach seems 
overly complex and potentially inefficient. Detailing the complexity, however, we have O (NlogN) for the two 
transforms (computed using the FFT algorithm) and O (NV) for the multiplication by the transfer function, 
which makes the total complexity O(NlogN) for N input values. A frequency domain implementation 
thus requires O (logN) computational complexity for each output value. The complexities of time-domain 
and frequency-domain implementations depend on different aspects of the filtering: The time-domain imple- 
mentation depends on the combined orders of the filter while the frequency-domain implementation depends 
on the logarithm of the Fourier transform’s length. 

It could well be that in some problems the time-domain version is more efficient (more easily satisfies the 
real time requirement), while in others the frequency domain approach is faster. In the latter situations, it is 
the FFT algorithm for computing the Fourier transforms that enables the superiority of frequency-domain 
implementations. Because complexity considerations only express how algorithm running-time increases with 
system parameter choices, we need to detail both implementations to determine which will be more suitable 
for any given filtering problem. Filtering with a difference equation is straightforward, and the number of 
computations that must be made for each output value is 2 (p + q). 


Exercise 5.34 (Solution on p. 194.) 
Derive this value for the number of computations for the general difference equation (5.42). 


30This content is available online at http: //cnx.org/content/m0511/2.21/. 


180 CHAPTER 5. DIGITAL SIGNAL PROCESSING 


5.17 Digital Signal Processing Problems” 


Problem 5.1: Sampling and Filtering 
The signal s(t) is bandlimited to 4 kHz. We want to sample it, but it has been subjected to various signal 
processing manipulations. 


(a) What sampling frequency (if any works) can be used to sample the result of passing s(t) through an 
RC highpass filter with R = 10kQ and C = 8nF? 

(b) What sampling frequency (if any works) can be used to sample the derivative of s(t)? 

(c) The signal s(t) has been modulated by an 8 kHz sinusoid having an unknown phase: the resulting 
signal is s (t) sin (27 fot + @), with fo = 8 kHz and ¢ =? Can the modulated signal be sampled so that 
the original signal can be recovered from the modulated signal regardless of the phase value ¢? If so, 
show how and find the smallest sampling rate that can be used; if not, show why not. 


Problem 5.2: Non-Standard Sampling 
Using the properties of the Fourier series can ease finding a signal’s spectrum. 


(a) Suppose a signal s (t) is periodic with period T. If cy, represents the signal’s Fourier series coefficients, 
what are the Fourier series coefficients of s (t - zt)? 
(b) Find the Fourier series of the signal p(t) shown in Figure 5.25. 


1 
(c) Suppose this signal is used to sample a signal bandlimited to a Hz. Find an expression for and sketch 
the spectrum of the sampled signal. 
(d) Does aliasing occur? If so, can a change in sampling rate prevent aliasing; if not, show how the signal 
can be recovered from these samples. 


Figure 5.25 


Problem 5.3: A Different Sampling Scheme 
A signal processing engineer from Texas A&M claims to have developed an improved sampling scheme. He 
multiplies the bandlimited signal by the depicted periodic pulse signal to perform sampling (Figure 5.26). 


31This content is available online at http: //cnx.org/content/m10351/2.38/. 
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Figure 5.26 


(a) Find the Fourier spectrum of this signal. 
(b) Will this scheme work? If so, how should Ts be related to the signal’s bandwidth? If not, why not? 


Problem 5.4: Bandpass Sampling 
The signal s(t) has the spectrum shown in Figure 5.27. 


S(f) 


PT 
Figure 5.27 


(a) What is the minimum sampling rate for this signal suggested by the Sampling Theorem? 
(b) Because of the particular structure of this spectrum, one wonders whether a lower sampling rate could 
be used. Show that this is indeed the case, and find the system that reconstructs s (t) from its samples. 


Problem 5.5: Sampling Signals 
1 
If a signal is bandlimited to W Hz, we can sample it at any rate — > 2W and recover the waveform exactly. 


This statement of the Sampling Theorem can be taken to mean that all information about the original 
signal can be extracted from the samples. While true in principle, you do have to be careful how you do 
so. In addition to the rms value of a signal, an important aspect of a signal is its peak value, which equals 


mas {|s (t) | }. 


(a) Let s(t) be a sinusoid having frequency W Hz. If we sample it at precisely the Nyquist rate, how 
accurately do the samples convey the sinusoid’s amplitude? In other words, find the worst case example. 

(b) How fast would you need to sample for the amplitude estimate to be within 5% of the true value? 

(c) Another issue in sampling is the inherent amplitude quantization produced by A/D converters. Assume 
the maximum voltage allowed by the converter is Vax volts and that it quantizes amplitudes to 
b bits. We can express the quantized sample Q(s(nT;)) as s(nT;) + €(t), where e(t) represents 
the quantization error at the n*® sample. Assuming the converter rounds, how large is maximum 
quantization error? 


182 CHAPTER 5. DIGITAL SIGNAL PROCESSING 


(d) We can describe the quantization error as noise, with a power proportional to the square of the 
maximum error. What is the signal-to-noise ratio of the quantization error for a full-range sinusoid? 
Express your result in decibels. 


Problem 5.6: Hardware Error 
An A/D converter has a curious hardware problem: Every other sampling pulse is half its normal amplitude 
(Figure 5.28). 


Figure 5.28 


(a) Find the Fourier series for this signal. 


(b) Can this signal be used to sample a bandlimited signal having highest frequency W = = 

Problem 5.7: Simple D/A Converter 

Commercial digital-to-analog converters don’t work this way, but a simple circuit illustrates how they work. 
Let’s assume we have a B-bit converter. Thus, we want to convert numbers having a B-bit representation 
into a voltage proportional to that number. The first step taken by our simple converter is to represent 
the number by a sequence of B pulses occurring at multiples of a time interval T. The presence of a pulse 
indicates a “1” in the corresponding bit position, and pulse absence means a “0” occurred. For a 4-bit 
converter, the number 13 has the binary representation 1101 (1319 = 1 x 2? +1 x 2?+0x 2'+1 x 2°) and 
would be represented by the depicted pulse sequence. Note that the pulse sequence is “backwards” from the 
binary representation. We’ll see why that is. 


A 
A e-a 
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Figure 5.29 


This signal (Figure 5.29) serves as the input to a first-order RC lowpass filter. We want to design the filter 
and the parameters A and T so that the output voltage at time 4T (for a 4-bit converter) is proportional 
to the number. This combination of pulse creation and filtering constitutes our simple D/A converter. The 
requirements are 
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e The voltage at time t = 4T should diminish by a factor of two the further the pulse occurs from this 
time. In other words, the voltage due to a pulse at 3T' should be twice that of a pulse produced at 2T, 
which in turn is twice that of a pulse at T,, etc. 

e The 4bit D/A converter must support a 10 kHz sampling rate. 


Show the circuit that works. How do the converter’s parameters change with sampling rate and number of 
bits in the converter? 


Problem 5.8: Discrete-Time Fourier Transforms 
Find the Fourier transforms of the following sequences, where s (n) is some sequence having Fourier transform 


S (eff). 


(a) (—1)"s(n) (b) s(n) cos (27 fon) 


(c) a(n) = i see (d) ns(n) 


0 n odd 


Problem 5.9: Spectra of Finite-Duration Signals 
Find the indicated spectra for the following signals. 


(a) The discrete-time Fourier transform of s(n) = 


a (Zn) n= {-1,0,1} 


0 otherwise 

n n= {-—2,-1,0,1,2} 
0 otherwise 

sin(Zn) n= {0,...,7} 
0 otherwise 


(b) The discrete-time Fourier transform of s(n) = 


(c) The discrete-time Fourier transform of s(n) = 
(d) The length-8 DFT of the previous signal. 


Problem 5.10: Just Whistlin’ 

Sammy loves to whistle and decides to record and analyze his whistling in lab. He is a very good whistler; his 
whistle is a pure sinusoid that can be described by s, (t) = sin (4000t). To analyze the spectrum, he samples 
his recorded whistle with a sampling interval of Ty = 2.5 x 10~* to obtain s(n) = s_ (nTs). Sammy (wisely) 
decides to analyze a few samples at a time, so he grabs 30 consecutive, but arbitrarily chosen, samples. He 
calls this sequence x (n) and realizes he can write it as 


x(n) = sin (4000nTs + 0), n = {0,..., 29} 


(a) Did Sammy under- or over-sample his whistle? 
(b) What is the discrete-time Fourier transform of x(n) and how does it depend on 6? 
(c) How does the 32-point DFT of x(n) depend on 6? 


Problem 5.11: Discrete-Time Filtering 
We can find the input-output relation for a discrete-time filter much more easily than for analog filters. The 
key idea is that a sequence can be written as a weighted linear combination of unit samples. 


(a) Show that x(n) = >>, x (i) 6 (n — 7) where 6 (n) is the unit-sample. 


= {1 n=0 


0 otherwise 


(b) If h(n) denotes the unit-sample response—the output of a discrete-time linear, shift-invariant filter 
to a unit-sample input—find an expression for the output. 
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(c) In particular, assume our filter is FIR, with the unit-sample response having duration q+ 1. If the 
input has duration N, what is the duration of the filter’s output to this signal? 


1 
(d) Let the filter be a boxcar averager: h(n) = aa for n = {0,...,q} and zero otherwise. Let the input 
q 
1 
be a pulse of unit height and duration N. Find the filter’s output when N = an q an odd integer. 


Problem 5.12: A Digital Filter 
A digital filter has the unit-sample response depicted in Figure 5.30. 


h(n) 


rn 


oO 
Te) 
AO 


Figure 5.30 


(a) What is the difference equation that defines this filter’s input-output relationship? 
(b) What is this filter’s transfer function? 


(c) What is the filter’s output when the input is sin (=) 


Problem 5.13: A Special Discrete-Time Filter 
Consider a FIR filter governed by the difference equation 


y(n) = 50(n4$2)+ Sa (nt1) +a(n) + S2(n 1) + 5a(n 2) 


(a) Find this filter’s unit-sample response. 

(b) Find this filter’s transfer function. Characterize this transfer function (i-e., what classic filter category 
does it fall into). 

(c) Suppose we take a sequence and stretch it out by a factor of three. 


em ={2) n=Bm,m={.5-,01--} 


0 otherwise 


Sketch the sequence x(n) for some example s(n). What is the filter’s output to this input? In 
particular, what is the output at the indices where the input x(n) is intentionally zero? Now how 
would you characterize this system? 


Problem 5.14: Simulating the Real World 

Much of physics is governed by differential equations, and we want to use signal processing methods to 
simulate physical problems. The idea is to replace the derivative with a discrete-time approximation and 
solve the resulting differential equation. For example, suppose we have the differential equation 


Su (t) + ay (t) =2() 
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and we approximate the derivative by 


where T essentially amounts to a sampling interval. 


(a) What is the difference equation that must be solved to approximate the differential equation? 

(b) When x(t) = u(t), the unit step, what will be the simulated output? 

(c) Assuming x (t) is a sinusoid, how should the sampling interval T be chosen so that the approximation 
works well? 


Problem 5.15: Derivatives 
The derivative of a sequence makes little sense, but still, we can approximate it. The digital filter described 
by the difference equation 


y(n) = a(n) —a(n—1) 


resembles the derivative formula. We want to explore how well it works. 


(a) What is this filter’s transfer function? 
(b) What is the filter’s output to the depicted triangle input (Figure 5.31)? 


Figure 5.31 


(c) Suppose the signal x(n) is a sampled analog signal: «(n) = #(nT;). Under what conditions will the 
filter act like a differentiator? In other words, when will y(n) be proportional to 42 (t) [eee 


Problem 5.16: The DFT 
Let’s explore the DFT and its properties. 


(a) What is the length-k DFT of length-N boxcar sequence, where N < K? 

(b) Consider the special case where K = 4. Find the inverse DFT of the product of the DFTs of two 
length-3 boxcars. 

(c) If we could use DFTs to perform linear filtering, it should be true that the product of the input’s DFT 
and the unit-sample response’s DFT equals the output’s DFT. So that you can use what you just 
calculated, let the input be a boxcar signal and the unit-sample response also be a boxcar. The result 
of part (b) would then be the filter’s output if we could implement the filter with length-4 DFTs. Does 
the actual output of the boxcar-filter equal the result found in the previous part (list, p. 185)? 

(d) What would you need to change so that the product of the DFTs of the input and unit-sample response 
in this case equaled the DFT of the filtered output? 
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Problem 5.17: DSP Tricks 

Sammy is faced with computing lots of discrete Fourier transforms. He will, of course, use the FFT algorithm, 
but he is behind schedule and needs to get his results as quickly as possible. He gets the idea of computing 
two transforms at one time by computing the transform of s(n) = s; (n) + js2 (nm), where s(n) and s2 (n) 
are two real-valued signals of which he needs to compute the spectra. The issue is whether he can retrieve 
the individual DFTs from the result or not. 


(a) What will be the DFT S(k) of this complex-valued signal in terms of S; (k) and $3 (k), the DFTs of 
the original signals? 

(b) Sammy’s friend, an Aggie who knows some signal processing, says that retrieving the wanted DFTs is 
easy: “Just find the real and imaginary parts of S (k).” Show that this approach is too simplistic. 

(c) While his friend’s idea is not correct, it does give him an idea. What approach will work? Hint: Use 
the symmetry properties of the DFT. 

(d) How does the number of computations change with this approach? Will Sammy’s idea ultimately lead 
to a faster computation of the required DFTs? 


Problem 5.18: Discrete Cosine Transform (DCT) 
The discrete cosine transform of a length-N sequence is defined to be 


Se(k) = > FO (= ) 


n=0 


Note that the number of frequency terms is 2N — 1: k = {0,...,2N — 1}. 


(a) Find the inverse DCT. 

(b) Does a Parseval’s Theorem hold for the DCT? 

(c) You choose to transmit information about the signal s(n) according to the DCT coefficients. You 
could only send one, which one would you send? 


Problem 5.19: A Digital Filter 
A digital filter is described by the following difference equation: 


y(n) = ay(n—1)+ aa (n)—a(n—-1), a= 


(a) What is this filter’s unit sample response? 
(b) What is this filter’s transfer function? 
(c) What is this filter’s output when the input is sin (4)? 


Problem 5.20: Another Digital Filter 
A digital filter is determined by the following difference equation. 


y(n) =y(n—1) +a(n)—2(n—4) 


(a) Find this filter’s unit sample response. 

(b) What is the filter’s transfer function? How would you characterize this filter (lowpass, highpass, special 
purpose, something else)? 

(c) Find the filter’s output when the input is the sinusoid sin (4). 

(d) In another case, the input sequence is zero for n < 0, then becomes nonzero. Sammy measures the 
output to be y(n) = 6(n) +6(n—1). Can his measurement be correct? In other words, is there an 
input that can yield this output? If so, find the input x(n) that gives rise to this output. If not, why 
not? 
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Problem 5.21: Yet Another Digital Filter 
A filter has an input-output relationship given by the following difference equation. 


y(n) = 70 (n) + 50(n 1) + =a (n — 2) 


(a) What is the filter’s transfer function? How would you characterize it? 
(b) What is the filter’s output when the input equals cos (2)? 
(c) What is the filter’s output when the input is the depicted discrete-time square wave (Figure 5.32)? 


x(n) 


Figure 5.32 


Problem 5.22: A Digital Filter in the Frequency Domain 
We have a filter with the transfer function 


AH (ie) =e J?"F cos (27 f) 
operating on the input signal x(n) = 6 (n) — 6 (n — 2) that yields the output y(n). 


(a) What is the filter’s unit-sample response? 
(b) What is the discrete-Fourier transform of the output? 
(c) What is the time-domain expression for the output? 


Problem 5.23: Digital Filters 

A discrete-time system is governed by the difference equation 

x(n) +a2(n—1) 
2 


y(n) =y(n—1) + 


(a) Find the transfer function for this system. 
(b) What is this system’s output when the input is sin (3)? 
(c) If the output is observed to be y(n) = 46 (n) + 6(n — 1), then what is the input? 


Problem 5.24: Digital Filtering 
A digital filter has an input-output relationship expressed by the difference equation 


_ #(n)+a(n—1)+2(n—-—2)+2(n—-3) 
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(a) Plot the magnitude and phase of this filter’s transfer function. 


b) What is this filter’s output when x(n) = cos ae + 2 sin a ? 
( Pp 5 
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Problem 5.25: Detective Work 
The signal x(n) equals 6 (n) — 6 (n — 1). 


(a) Find the length-8 DFT (discrete Fourier transform) of this signal. 

(b) You are told that when x(n) served as the input to a linear FIR (finite impulse response) filter, the 
output was y(n) = 6(n) — 6(n— 1) + 26 (n — 2). Is this statement true? If so, indicate why and find 
the system’s unit sample response; if not, show why not. 


Problem 5.26: 
A discrete-time, shift invariant, linear system produces an output y(n) = {1,—1,0,0,...} when its input 
x(n) equals a unit sample. 


(a) Find the difference equation governing the system. 
(b) Find the output when x (n) = cos (27 fon). 
(c) How would you describe this system’s function? 


Problem 5.27: Time Reversal has Uses 

A discrete-time system has transfer function H (e/?"/). A signal x(n) is passed through this system to 
yield the signal w(n). The time-reversed signal w(—n) is then passed through the system to yield the 
time-reversed output y(—n). What is the transfer function between x(n) and y(n)? 


Problem 5.28: Removing “Hum” 

The slang word “hum” represents power line waveforms that creep into signals because of poor circuit con- 
struction. Usually, the 60 Hz signal (and its harmonics) are added to the desired signal. What we seek are 
filters that can remove hum. In this problem, the signal and the accompanying hum have been sampled; we 
want to design a digital filter for hum removal. 


(a) Find filter coefficients for the length-3 FIR filter that can remove a sinusoid having digital frequency 
fo from its input. 

(b) Assuming the sampling rate is f, to what analog frequency does fp correspond? 

(c) A more general approach is to design a filter having a frequency response magnitude proportional to 
the absolute value of a cosine: |H (e?"/) | x |cos(7fN)|. In this way, not only can the fundamental 
but also its first few harmonics be removed. Select the parameter N and the sampling rate so that the 
frequencies at which the cosine equals zero correspond to 60 Hz and its odd harmonics through the 
fifth. 

(d) Find the difference equation that defines this filter. 


Problem 5.29: Digital AM Receiver 

Thinking that digital implementations are always better, our clever engineer wants to design a digital AM 
receiver. The receiver would bandpass the received signal, pass the result through an A/D converter, perform 
all the demodulation with digital signal processing systems, and end with a D/A converter to produce the 
analog message signal. Assume in this problem that the carrier frequency is always a large even multiple of 
the message signal’s bandwidth W. 


(a) What is the smallest sampling rate that would be needed? 

(b) Show the block diagram of the least complex digital AM receiver. 

(c) Assuming the channel adds white noise and that a b-bit A/D converter is used, what is the output’s 
signal-to-noise ratio? 


Problem 5.30: DFTs 
A problem on Samantha’s homework asks for the 8-point DFT of the discrete-time signal 6 (n — 1)+6 (n — 7). 
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(a) What answer should Samantha obtain? 

(b) As a check, her group partner Sammy says that he computed the inverse DFT of her answer and got 
d(n+1)+6(n—1). Does Sammy’s result mean that Samantha’s answer is wrong? 

(c) The homework problem says to lowpass-filter the sequence by multiplying its DFT by 


re f k = {0,1,7} 


0 otherwise 


and then computing the inverse DFT. Will this filtering algorithm work? If so, find the filtered output; 
if not, why not? 


Problem 5.31: Stock Market Data Processing 

Because a trading week lasts five days, stock markets frequently compute running averages each day over the 
previous five trading days to smooth price fluctuations. The technical stock analyst at the Buy-Lo-Sell-Hi 
brokerage firm has heard that FFT filtering techniques work better than any others (in terms of producing 
more accurate averages). 


(a) What is the difference equation governing the five-day averager for daily stock prices? 

(b) Design an efficient FFT-based filtering algorithm for the broker. How much data should be processed 
at once to produce an efficient algorithm? What length transform should be used? 

(c) Is the analyst’s information correct that FFT techniques produce more accurate averages than any 
others? Why or why not? 


Problem 5.32: Echoes 
Echoes not only occur in canyons, but also in auditoriums and telephone circuits. In one situation where 
the echoed signal has been sampled, the input signal x (n) emerges as x (n) + a,% (n — 21) + aga (n — ng). 


(a) Find the difference equation of the system that models the production of echoes. 

(b) To simulate this echo system, ELEC 241 students are asked to write the most efficient (quickest) 
program that has the same input-output relationship. Suppose the duration of x(n) is 1,000 and that 
ay 3 ny = 10, ag z, and nz = 25. Half the class votes to just program the difference equation 
while the other half votes to program a frequency domain approach that exploits the speed of the FFT. 
Because of the undecided vote, you must break the tie. Which approach is more efficient and why? 

(c) Find the transfer function and difference equation of the system that suppresses the echoes. In other 


words, with the echoed signal as the input, what system’s output is the signal x (n)? 


Problem 5.33: Digital Filtering of Analog Signals 
RU Electronics wants to develop a filter that would be used in analog applications, but that is implemented 
digitally. The filter is to operate on signals that have a 10 kHz bandwidth, and will serve as a lowpass filter. 


(a) What is the block diagram for your filter implementation? Explicitly denote which components are 
analog, which are digital (a computer performs the task), and which interface between analog and 
digital worlds. 

(b) What sampling rate must be used and how many bits must be used in the A/D converter for the 
acquired signal’s signal-to-noise ratio to be at least 60 dB? For this calculation, assume the signal is a 
sinusoid. 

(c) If the filter is a length-128 FIR filter (the duration of the filter’s unit-sample response equals 128), 
should it be implemented in the time or frequency domain? 

(d) Assuming H (e/?/) is the transfer function of the digital filter, what is the transfer function of your 
system? 


Problem 5.34: Signal Compression 
Because of the slowness of the Internet, lossy signal compression becomes important if you want signals to 
be received quickly. An enterprising ELEC 241 student has proposed a scheme based on frequency-domain 
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processing. First of all, he would section the signal into length-N blocks, and compute its N-point DFT. He 
then would discard (zero the spectrum) at half of the frequencies, quantize them to b-bits, and send these 
over the network. The receiver would assemble the transmitted spectrum and compute the inverse DFT, 
thus reconstituting an N-point block. 


(a) At what frequencies should the spectrum be zeroed to minimize the error in this lossy compression 
scheme? 

(b) The nominal way to represent a signal digitally is to use simple b-bit quantization of the time-domain 
waveform. How long should a section be in the proposed scheme so that the required number of 
bits/sample is smaller than that nominally required? 

(c) Assuming that effective compression can be achieved, would the proposed scheme yield satisfactory 
results? 
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Solutions to Exercises in Chapter 5 


Solution to Exercise 5.1 (p. 145) 

For b-bit signed integers, the largest number is 2°~! — 1. For b = 32, we have 2,147,483,647 and for b = 64, 
we have 9,223,372,036,854,775,807 or about 9.2 x 101°. 

Solution to Exercise 5.2 (p. 146) 

In floating point, the number of bits in the exponent determines the largest and smallest representable 
numbers. For 32-bit floating point, the largest (smallest) numbers are 2+!27 = 1.7 x 10°° (5.9 x 107°). For 
64-bit floating point, the largest number is about 10°°°?. 

Solution to Exercise 5.3 (p. 146) 

25 = 110112 and 7 = 1119. We find that 110012 + 1112 = 1000002 = 32. 

Solution to Exercise 5.4 (p. 148) 

The only effect of pulse duration is to unequally weight the spectral repetitions. Because we are only 
concerned with the repetition centered about the origin, the pulse duration has no significant effect on 
recovering a signal from its samples. 

Solution to Exercise 5.5 (p. 149) 


Figure 5.33 


The square wave’s spectrum is shown by the bolder set of lines centered about the origin. The dashed 
lines correspond to the frequencies about which the spectral repetitions (due to sampling with T, = 1) 
occur. As the square wave’s period decreases, the negative frequency lines move to the left and the positive 
frequency ones to the right. 

Solution to Exercise 5.6 (p. 149) 

The simplest bandlimited signal is the sine wave. At the Nyquist frequency, exactly two samples/period 
would occur. Reducing the sampling rate would result in fewer samples/period, and these samples would 
appear to have arisen from a lower frequency sinusoid. 

Solution to Exercise 5.7 (p. 150) 

The plotted temperatures were quantized to the nearest degree. Thus, the high temperature’s amplitude 
was quantized as a form of A/D conversion. 

Solution to Exercise 5.8 (p. 151) 

The signal-to-noise ratio does not depend on the signal amplitude. With an A/D range of [—A, A], the 


9B 
Solution to Exercise 5.9 (p. 151) 
Solving 2~? = .001 results in B = 10 bits. 


quantization interval A = and the signal’s rms value (again assuming it is a sinusoid) is A/V/2. 
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Solution to Exercise 5.10 (p. 151) 
A 16-bit A/D converter yields a SNR of 6 x 16 + 10log,91.5 = 97.8 dB. 
Solution to Exercise 5.11 (p. 154) 


CoO 
8 (2? F+D) = So s(n) eo2mftin 
n=—oo 
[oe) 
= S- ce J2mN g (n) eden fn 
n=—0o (5.56) 
[oe) 
= S- s(n)e Irn 
n=—0oo 
=$ (ae) 
Solution to Exercise 5.12 (p. 155) 
N+no-1 N+no-1 
a S- a” — S- eo = aN tno — qr 
nN=No nN=No 


which, after manipulation, yields the geometric sum formula. 

Solution to Exercise 5.13 (p. 158) 

If the sampling frequency exceeds the Nyquist frequency, the spectrum of the samples equals the analog 
spectrum, but over the normalized analog frequency fT. Thus, the energy in the sampled signal equals the 
original signal’s energy multiplied by T. 

Solution to Exercise 5.14 (p. 158) 

After lowpass filtering, the interspersed zeros at the odd indices would be filled in with signal values that 
correspond to the original analog signal sampled at twice the rate. Consequently, “stretching” the signal by 
a factor of M then lowpass filtering with a cutoff frequency of 1/2M corresponds to up-sampling (a virtual 
increase of the sampling rate) by a factor of M. 

Solution to Exercise 5.15 (p. 160) 

This situation amounts to aliasing in the time-domain. 

Solution to Exercise 5.16 (p. 161) 

Since conjugate symmetry requires S'(N/2) = S*(N/2), the spectrum at the “halfway” index must be real- 
valued. 

Solution to Exercise 5.17 (p. 162) 

When the signal is real-valued, we may only need half the spectral values, but the complexity remains 
unchanged. If the data are complex-valued, which demands retaining all frequency values, the complexity is 
again the same. When only K frequencies are needed, the complexity is O (KN). 

Solution to Exercise 5.18 (p. 162) 

If a DFT required 1 ms to compute, and signal having ten times the duration would require 100ms to 
compute. Using the FFT, a lms computing time would increase by a factor of about 10 log, 10 = 33, a 
factor of three less than the DFT would have needed. 

Solution to Exercise 5.19 (p. 164) 

The upper panel has not used the FFT algorithm to compute the length-4 DFTs while the lower one has. 
The ordering is determined by the algorithm. 

Solution to Exercise 5.20 (p. 164) 

The transform can have any greater than or equal to the actual duration of the signal. We simply “pad” the 
signal with zero-valued samples until a computationally advantageous signal length results. Recall that the 
FFT is an algorithm to compute the DFT. Extending the length of the signal this way merely means we 
are sampling the frequency axis more finely than required. To use the Cooley-Tukey algorithm, the length 
of the resulting zero-padded signal can be 512, 1024, etc. samples long. 
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Solution to Exercise 5.21 (p. 164) 

Number of samples equals 1.2 x 11025 = 13230. The datarate is 11025 x 16 = 176.4 kbps. The storage 
required would be 26460 bytes. 

Solution to Exercise 5.22 (p. 165) 

The oscillations are due to the boxcar window’s Fourier transform, which equals the sinc function. 
Solution to Exercise 5.23 (p. 167) 

These numbers are powers-of-two, and the FFT algorithm can be exploited with these lengths. To compute 
a longer transform than the input signal’s duration, we simply zero-pad the signal. 

Solution to Exercise 5.24 (p. 167) 

In discrete-time signal processing, an amplifier amounts to a multiplication, a very easy operation to perform. 
Solution to Exercise 5.25 (p. 169) 

The indices can be negative, and this condition is not allowed in MATLAB. To fix it, we must start the 
signals later in the array. 

Solution to Exercise 5.26 (p. 170) 

Such terms would require the system to know what future input or output values would be before the current 
value was computed. Thus, such terms can cause difficulties. 

Solution to Exercise 5.27 (p. 172) 

It now acts like a bandpass filter with a center frequency of fp and a bandwidth equal to twice of the original 
lowpass filter. 

Solution to Exercise 5.28 (p. 172) 

The DTFT of the unit sample equals a constant (equaling 1). Thus, the Fourier transform of the output 
equals the transfer function. 

Solution to Exercise 5.29 (p. 173) 

In sampling a discrete-time signal’s Fourier transform DL times equally over [0,27) to form the DFT, the 
corresponding signal equals the periodic repetition of the original signal. 


S(k) <> 3 s(n —iL) (5.57) 


1=— 00 


To avoid aliasing (in the time domain), the transform length must equal or exceed the signal’s duration. 
Solution to Exercise 5.30 (p. 173) 
The difference equation for an FIR filter has the form 


y(n) = > bmx (n — m) (5.58) 
m=0 


The unit-sample response equals 
q 
h(n) = S~ bd (n — m) (5.59) 
m=0 


which corresponds to the representation described in a problem (Example 5.6) of a length-qg boxcar filter. 
Solution to Exercise 5.31 (p. 173) 

The unit-sample response’s duration is g+ 1 and the signal’s N,. Thus the statement is correct. 

Solution to Exercise 5.32 (p. 177) 

Let N denote the input’s total duration. The time-domain implementation requires a total of N (2q+ 1) 


computations, or 2q¢+ 1 computations per input value. In the frequency domain, we split the input into — 
q ie 
Na+ 
by N,, to find the number of computations per input value in the entire input, this quantity decreases as 
N, increases. For the time-domain implementation, it stays constant. 


sections, each of which requires log, (Nz + q) +6+ per input in the section. Because we divide again 
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Solution to Exercise 5.33 (p. 178) 

The delay is not computational delay here—the plot shows the first output value is aligned with the filter’s first 
input—although in real systems this is an important consideration. Rather, the delay is due to the filter’s phase 
shift: A phase-shifted sinusoid is equivalent to a time-delayed one: cos (2a fn — ¢) = cos (2x f (n — z+). 
All filters have phase shifts. This delay could be removed if the filter introduced no phase shift. Such filters 
do not exist in analog form, but digital ones can be programmed, but not in real time. Doing so would 
require the output to emerge before the input arrives! 

Solution to Exercise 5.34 (p. 179) 

We have p+ q+1 multiplications and p+ q-— 1 additions. Thus, the total number of arithmetic operations 


equals 2 (p+ q). 


Chapter 6 


Information Communication 


6.1 Information Communication’ 


As far as a communications engineer is concerned, signals express information. Because systems manipulate 
signals, they also affect the information content. Information comes neatly packaged in both analog and 
digital forms. Speech, for example, is clearly an analog signal, and computer files consist of a sequence 
of bytes, a form of “discrete-time” signal despite the fact that the index sequences byte position, not time 
sample. Communication systems endeavor not to manipulate information, but to transmit it from one 
place to another, so-called point-to-point communication, from one place to many others, broadcast 
communication, or from many to many, like a telephone conference call or a chat room. Communication 
systems can be fundamentally analog, like radio, or digital, like computer networks. 

This chapter develops a common theory that underlies how such systems work. We describe and analyze 
several such systems, some old like AM radio, some new like computer networks. The question as to which is 
better, analog or digital communication, has been answered, because of Claude Shannon’s fundamental work 
on a theory of information published in 1948, the development of cheap, high-performance computers, and 
the creation of high-bandwidth communication systems. The answer is to use a digital communication 
strategy. In most cases, you should convert all information-bearing signals into discrete-time, amplitude- 
quantized signals. Fundamentally digital signals, like computer files (which are a special case of symbolic 
signals), are in the proper form. Because of the Sampling Theorem, we know how to convert analog signals 
into digital ones. Shannon showed that once in this form, a properly engineered system can communi- 
cate digital information with no error despite the fact that the communication channel thrusts 
noise onto all transmissions. This startling result has no counterpart in analog systems; AM radio 
will remain noisy. The convergence of these theoretical and engineering results on communications systems 
has had important consequences in other arenas. The audio compact disc (CD) and the digital videodisk 
(DVD) are now considered digital communications systems, with communication design considerations used 
throughout. 

Go back to the fundamental model of communication (Figure 1.3). Communications design begins with 
two fundamental considerations. 


1. What is the nature of the information source, and to what extent can the receiver tolerate errors in 
the received information? 
2. What are the channel’s characteristics and how do they affect the transmitted signal? 


In short, what are we going to send and how are we going to send it? Interestingly, digital as well as 
analog transmission are accomplished using analog signals, like voltages in Ethernet (an example of wireline 
communications) and electromagnetic radiation (wireless) in cellular telephone. 


1This content is available online at http: //cnx.org/content/m0513/2.8/. 
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6.2 Types of Communication Channels’ 


Electrical communications channels are either wireline or wireless channels. Wireline channels physically 
connect transmitter to receiver with a “wire” which could be a twisted pair, coaxial cable or optic fiber. 
Consequently, wireline channels are more private and much less prone to interference. Simple wireline 
channels connect a single transmitter to a single receiver: a point-to-point connection as with the telephone. 
Listening in on a conversation requires that the wire be tapped and the voltage measured. Some wireline 
channels operate in broadcast modes: one or more transmitter is connected to several receivers. One simple 
example of this situation is cable television. Computer networks can be found that operate in point-to-point 
or in broadcast modes. Wireless channels are much more public, with a transmitter’s antenna radiating 
a signal that can be received by any antenna sufficiently close enough. In contrast to wireline channels 
where the receiver takes in only the transmitter’s signal, the receiver’s antenna will react to electromagnetic 
radiation coming from any source. This feature has two faces: The smiley face says that a receiver can take 
in transmissions from any source, letting receiver electronics select wanted signals and disregarding others, 
thereby allowing portable transmission and reception, while the frowny face says that interference and noise 
are much more prevalent than in wireline situations. A noisier channel subject to interference compromises 
the flexibility of wireless communication. 

Maxwell’s equations® neatly summarize the physics of all electromagnetic phenomena, including cir- 
cuits, radio, and optic fiber transmission. 


Vixens! Ga) div (cE) = p 
ot , (6.1) 
VxH=oE+ @ (cE) div (uH) = 0 


where E is the electric field, H the magnetic field, € dielectric permittivity, 4 magnetic permeability, o 
electrical conductivity, and p is the charge density. Kirchhoff’s Laws represent special cases of these equations 
for circuits. We are not going to solve Maxwell’s equations here; do bear in mind that a fundamental 
understanding of communications channels ultimately depends on fluency with Maxwell’s equations. Perhaps 
the most important aspect of them is that they are linear with respect to the electrical and magnetic fields. 
Thus, the fields (and therefore the voltages and currents) resulting from two or more sources will add. 


NOTE: Nonlinear electromagnetic media do exist. The equations as written here are simpler 
versions that apply to free-space propagation and conduction in metals. Nonlinear media are be- 
coming increasingly important in optic fiber communications, which are also governed by Maxwell’s 
equations. 


6.3 Wireline Channels’ 


Wireline channels were the first used for electrical communications in the mid-nineteenth century for the 
telegraph. Here, the channel is one of several wires connecting transmitter to receiver. The transmitter 
simply creates a voltage related to the message signal and applies it to the wire(s). We must have a circuit— 
a closed path—that supports current flow. In the case of single-wire communications, the earth is used as the 
current’s return path. In fact, the term ground for the reference node in circuits originated in single-wire 
telegraphs. You can imagine that the earth’s electrical characteristics are highly variable, and they are. 
Single-wire metallic channels cannot support high-quality signal transmission having a bandwidth beyond a 
few hundred Hertz over any appreciable distance. 

Consequently, most wireline channels today essentially consist of pairs of conducting wires (see Figure 6.1), 
and the transmitter applies a message-related voltage across the pair. How these pairs of wires are physically 
configured greatly affects their transmission characteristics. One example is twisted pair, wherein the wires 


?This content is available online at http: //cnx.org/content/m0099/2.13/. 

3James Clerk Maxwell’s biography can be found at http://www-groups.dcs.st-andrews.ac.uk/~history/Biographies/ 
Maxwell.html. 

“This content is available online at http: //cnx.org/content/m0100/2.29/. 
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Figure 6.1: Coaxial cable consists of one conductor wrapped around the central conductor. This type 
of cable supports broader bandwidth signals than twisted pair, and finds use in cable television and 


Ethernet. 
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o> 00 o> TM O> 
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- V(x-Ax) GAxS CAx== V(x) GAx<S, CAx== V(x+Ax) *° 
6 6 5 


Figure 6.2: The so-called distributed parameter model for two-wire cables has the depicted circuit 
model structure. Element values depend on geometry and the properties of materials used to construct 
the transmission line. 


are wrapped about each other. Telephone cables are one example of a twisted pair channel. Another is 
coaxial cable, where a concentric conductor surrounds a central wire with a dielectric material in between. 
Coaxial cable, fondly called “co-ax” by engineers, is what Ethernet uses as its channel. In either case, wireline 
channels form a dedicated circuit between transmitter and receiver. As we shall find subsequently, several 
transmissions can share the circuit by amplitude modulation techniques; commercial cable TV is an example. 
These information-carrying circuits are designed so that interference from nearby electromagnetic sources is 
minimized. Thus, by the time signals arrive at the receiver, they are relatively interference- and noise-free. 

Both twisted pair and co-ax are examples of transmission lines, which all have the circuit model 
shown in Figure 6.2 for an infinitesimally small length. This circuit model arises from solving Maxwell’s 
equations for the particular transmission line geometry. The series resistance comes from the conductor 
used in the wires and from the conductor’s geometry. The inductance and the capacitance derive from 
transmission line geometry, and the parallel conductance from the medium between the wire pair. Note 
that all the circuit elements have values expressed by the product of a constant times a length; this notation 
represents that element values here have per-unit-length units. For example, the series resistance R has units 
of ohms/meter. For coaxial cable, the element values depend on the inner conductor’s radius r;, the outer 
radius of the dielectric rg, the conductivity of the conductors a, and the conductivity og, dielectric constant 
€q, and magnetic permittivity gq of the dielectric as 


~ 1 1 1 ~ 27 €q 
fod) Gee 
~ 2104 ~~ phd Td 62) 
G= in (2) L= 3, (**) 
Ti 
For twisted pair, having a separation d between the conductors that have conductivity 0 and common radius 
r and that are immersed in a medium having dielectric and magnetic properties, the element values are then 
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~ 1 ~ TE 
a Ge) 


a TO oh Oe, (ad 
Geek) Z= 4 (+ cos (=)) 


The voltage between the two conductors and the current flowing through them will depend on distance x 
along the transmission line as well as time. We express this dependence as v (x,t) and i (a,t). When we place 
a sinusoidal source at one end of the transmission line, these voltages and currents will also be sinusoidal 
because the transmission line model consists of linear circuit elements. As is customary in analyzing linear 
circuits, we express voltages and currents as the real part of complex exponential signals, and write circuit 
variables as a complex amplitude—here dependent on distance—times a complex exponential: v (x,t) = 
Re [V (x) e??"/"] and i(az,t) = Re [I (x) e/?"/"]. Using the transmission line circuit model, we find from 
KCL, KVL, and v-i relations the equations governing the complex amplitudes. 

KCL at Center Node 


(6.3) 


I () =I (e— Ax) —V (a) (G+ janfC) Ac (6.4) 


V-I relation for RL series 
V (x) —V (« + Az) = I (2) (B+ j2nf i) Ac (6.5) 


Rearranging and taking the limit Az — 0 yields the so-called transmission line equations. 


Jel (2) =~ (G+ 52 fC) Va) (6.6) 


—V (#2) =- (R + janfL) I(x) 


By combining these equations, we can obtain a single equation that governs how the voltage’s or the current’s 
complex amplitude changes with position along the transmission line. Taking the derivative of the second 
equation and plugging the first equation into the result yields the equation governing the voltage. 


Vi = (G + j2nfC) (R + j2nfL) V (a) (6.7) 
This equation’s solution is 
V (z) =Vie"17 + V_et™ (6.8) 


Calculating its second derivative and comparing the result with our equation for the voltage can check this 
solution. 


d? 2 
aio = yx +yx 
Te Ve ee) (6.9) 
=7°V(z) 
Our solution works so long as the quantity y satisfies 
7 +/( 4: jong) (f a jonfL) 6 


= +(a(f) + 3b(f)) 


Thus, y depends on frequency, and we express it in terms of real and imaginary parts as indicated. The 
quantities V; and V_ are constants determined by the source and physical considerations. For example, 
let the spatial origin be the middle of the transmission line model Figure 6.2. Because the circuit model 
contains simple circuit elements, physically possible solutions for voltage amplitude cannot increase with 
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distance along the transmission line. Expressing 7 in terms of its real and imaginary parts in our solution 
shows that such increases are a (mathematical) possibility. V (7) = Vpe~(¢+99)® + V_et+(¢+0)" The voltage 
cannot increase without limit; because a(f) is always positive, we must segregate the solution for negative 
and positive x. The first term will increase exponentially for x < 0 unless Vi = 0 in this region; a similar 
result applies to V_ for x > 0. These physical constraints give us a cleaner solution. 
—(a+ jb)x : 
via {Pr es a ae (6.11) 
Viet@ts)= ife<0 


This solution suggests that voltages (and currents too) will decrease exponentially along a transmission 
line. The space constant, also known as the attenuation constant, is the distance over which the voltage 
decreases by a factor of i, It equals the reciprocal of a(f), which depends on frequency, and is expressed by 
manufacturers in units of dB/m. 

The presence of the imaginary part of y, b(f), also provides insight into how transmission lines work. 
Because the solution for x > 0 is proportional to e~J°”, we know that the voltage’s complex amplitude will 
vary sinusoidally in space. The complete solution for the voltage has the form 


u(a,t) = Re Weer ene) (6.12) 


The complex exponential portion has the form of a propagating wave. If we could take a snapshot of the 
voltage (take its picture at ¢t = t,), we would see a sinusoidally varying waveform along the transmission 
line. One period of this variation, known as the wavelength, equals \ = 27/b. If we were to take a second 
picture at some later time t = tz, we would also see a sinusoidal voltage. Because 


2 
Qn fto — bu = 2nrf (t) + to —t1) — ba = Ar ft, —b (« wt (ta n)) 
the second waveform appears to be the first one, but delayed—shifted to the right—in space. Thus, the 
voltage appeared to move to the right with a speed equal to 27f/b (assuming b > 0). We denote this 
propagation speed by c, and it equals 


(6.13) 


art - 2n f 
m NG + j2nfC) (#+2ns2)| 


The characteristics of the voltage signal shown in equation (6.12) depend on the values of a and b, and 
how they depend on frequency. The simplest results occur in the high-frequency region where j27 f L>R 


and j27 pias > G. In this case, y simplifies to \/—41? f2Lc, which seemingly makes it pure imaginary with 
a=Oand b=2rfv Le, Using this result, we find the propagation speed to be 


; 1 
Pe Tig 
For typical coaxial cable, this propagation speed is a fraction (one-third to two-thirds) of the speed of light. 
While this high-frequency analysis shows that the dominant high-frequency component of 7¥ is its imagi- 
nary part, there could be (and is!) a smaller real part. Since the real part of y is the attenuation factor a, a 
more detailed analysis is required to determine if a = 0 (no attenuation) or is non-zero. One way of pursuing 
a more detailed analysis is to exploit equation (6.10) by 


(6.14) 


= (a+ jb)? = (G+ j2nfC) (B+ j2ns1) 
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Expanding the expressions, we find that 
a? —b? + j2ab = GR — 4n* f? LC + j20f(RC + GL) 


Considering the high-frequency limit, the constant term on the right side can be ignored. Setting b = 
Inf VLC and equating j2ab with j27f(RC + GL), we find that the real part indeed equals a non-zero 


constant. _ 
1/R ~ 
=-|— Z 
a 5 (Z +G 7 


Here, Zp is defined to be 4 Te / C and its importance is demonstrated below. The exponential attenuation of 
high-quality coaxial cable predicted by this result is typically 1 dB (or less) per kilometer. 
Exercise 6.1 (Solution on p. 255.) 
Find the propagation speed in terms of physical parameters for both the coaxial cable and twisted 
pair examples. 


By using the second of the transmission line equation (6.6), we can solve for the current’s complex 
amplitude. Considering the spatial region x > 0, for example, we find that 


£V (0) =—aW (a) = — (B+ s2nfF) 1(0) 


which means that the ratio of voltage and current complex amplitudes does not depend on distance. 


T(a)  \ G+ jonfe (6.15) 
= Zo 


The quantity Zp is known as the transmission line’s characteristic impedance. Note that when the signal 
frequency is sufficiently high, the characteristic impedance is real, which means the transmission line appears 
resistive in this high-frequency regime. 
Ra (6.16) 
fra C 
Typical values for characteristic impedance are 50 and 75 (2. 

A related transmission line is the optic fiber. Here, the electromagnetic field is light, and it propagates 
down a cylinder of glass. In this situation, we don’t have two conductors—in fact we have none—and the 
energy is propagating in what corresponds to the dielectric material of the coaxial cable. Optic fiber com- 
munication has exactly the same properties as other transmission lines: Signal strength decays exponentially 
according to the fiber’s space constant and propagates at some speed less than light would in free space. 
From the encompassing view of Maxwell’s equations, the only difference is the electromagnetic signal’s fre- 
quency. Because no electric conductors are present and the fiber is protected by an opaque “insulator,” optic 
fiber transmission is interference-free. 


Exercise 6.2 (Solution on p. 255.) 
From tables of physical constants, find the frequency of a sinusoid in the middle of the visible light 
range. Compare this frequency with that of a mid-frequency cable television signal. 


To summarize, we use transmission lines for high-frequency wireline signal communication. In wireline 
communication, we have a direct, physical connection—a circuit—between transmitter and receiver. When 
we select the transmission line characteristics and the transmission frequency so that we operate in the high- 
frequency regime, signals are not filtered as they propagate along the transmission line: The characteristic 
impedance is real-valued—the transmission line’s equivalent impedance is a resistor—and all the signal’s 
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components at various frequencies propagate at the same speed. Transmitted signal amplitude does decay 
exponentially along the transmission line. Note that in the high-frequency regime the space constant is small, 
which means the signal attenuates little. 


6.4 Wireless Channels’ 


Wireless channels exploit the prediction made by Maxwell’s equation that electromagnetic fields propagate 
in free space like light. When a voltage is applied to an antenna, it creates an electromagnetic field that 
propagates in all directions (although antenna geometry affects how much power flows in any given direction) 
that induces electric currents in the receiver’s antenna. Antenna geometry determines how energetic a field 
a voltage of a given frequency creates. In general terms, the dominant factor is the relation of the antenna’s 
size to the field’s wavelength. The fundamental equation relating frequency and wavelength for a propagating 
wave is 


Af =c 


Thus, wavelength and frequency are inversely related: High frequency corresponds to small wavelengths. 
For example, a 1 MHz electromagnetic field has a wavelength of 300 m. Antennas having a size or distance 
from the ground comparable to the wavelength radiate fields most efficiently. Consequently, the lower the 
frequency the bigger the antenna must be. Because most information signals are baseband signals, having 
spectral energy at low frequencies, they must be modulated to higher frequencies to be transmitted over 
wireless channels. 

For most antenna-based wireless systems, how the signal diminishes as the receiver moves further from 
the transmitter derives by considering how radiated power changes with distance from the transmitting 
antenna. An antenna radiates a given amount of power into free space, and ideally this power propagates 
without loss in all directions. Considering a sphere centered at the transmitter, the total power, which is 
found by integrating the radiated power over the surface of the sphere, must be constant regardless of the 
sphere’s radius. This requirement results from the conservation of energy. Thus, if p(d) represents the power 
integrated with respect to direction at a distance d from the antenna, the total power will be p(d) 47d?. For 
this quantity to be a constant, we must have 


which means that the received signal amplitude Ap must be proportional to the transmitter’s amplitude Ar 
and inversely related to distance from the transmitter. 


Apis (6.17) 


for some value of the constant k&. Thus, the further from the transmitter the receiver is located, the weaker 
the received signal. Whereas the attenuation found in wireline channels can be controlled by physical 
parameters and choice of transmission frequency, the inverse-distance attenuation found in wireless channels 
persists across all frequencies. 
Exercise 6.3 (Solution on p. 255.) 
Why don’t signals attenuate according to the inverse-square law in a conductor? What is the 
difference between the wireline and wireless cases? 
The speed of propagation is governed by the dielectric constant go and magnetic permeability €9 of free 
space. 
1 
V Ho€eo (6.18) 
= 3x 10°m/s 


5This content is available online at http: //cnx.org/content/m0101/2.15/. 
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Known familiarly as the speed of light, it sets an upper limit on how fast signals can propagate from one 
place to another. Because signals travel at a finite speed, a receiver senses a transmitted signal only after a 
time delay inversely related to the propagation speed: 


At =-—- 
c 
At the speed of light, a signal travels across the United States in 16 ms, a reasonably small time delay. If a 
lossless (zero space constant) coaxial cable connected the East and West coasts, this delay would be two to 
three times longer because of the slower propagation speed. 


6.5 Line-of-Sight Transmission’ 


Long-distance transmission over either kind of channel encounters attenuation problems. Losses in wireline 
channels can be compensated by repeaters can extend the distance between transmitter and receiver beyond 
what passive losses the wireline channel imposes. In wireless channels, not only does radiation loss occur 
(p. 201), but also one antenna may not “see” another because of the earth’s curvature. 


M« dios >i 


Figure 6.3: Two antennae are shown each having the same height. Line-of-sight transmission means 
the transmitting and receiving antennae can “see” each other as shown. The maximum distance at which 
they can see each other, dios, occurs when the sighting line just grazes the earth’s surface. 


At the usual radio frequencies, propagating electromagnetic energy does not follow the earth’s surface. 
Line-of-sight communication has the transmitter and receiver antennas in visual contact with each other. 
Assuming both antennas have height h above the earth’s surface, maximum line-of-sight distance is 


dros = 2\/2hAR +h? = 2V2Rh (6.19) 


where R is the earth’s radius ( 6.38 x 10°m). 


Exercise 6.4 (Solution on p. 255.) 
Derive the expression of line-of-sight distance using only the Pythagorean Theorem. Generalize it to 
the case where the antennas have different heights (as is the case with commercial radio and cellular 
telephone). What is the range of cellular telephone where the handset antenna has essentially zero 
height? 

Exercise 6.5 (Solution on p. 255.) 
Can you imagine a situation wherein global wireless communication is possible with only one trans- 
mitting antenna? In particular, what happens to wavelength when carrier frequency decreases? 


Using a 100 m antenna would provide line-of-sight transmission over a distance of 71.4 km. Using such very 
tall antennas would provide wireless communication within a town or between closely spaced population 


®This content is available online at http: //cnx.org/content/m0538/2.14/. 
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centers. Consequently, networks of antennas sprinkle the countryside (each located on the highest hill 
possible) to provide long-distance wireless communications: Each antenna receives energy from one antenna 
and retransmits to another. This kind of network is known as a relay network. 


6.6 The Ionosphere and Communications’ 


If we were limited to line-of-sight communications, long distance wireless communication, like ship-to-shore 
communication, would be impossible. At the turn of the century, Marconi, the inventor of wireless telegraphy, 
boldly tried such long distance communication without any evidence — either empirical or theoretical — 
that it was possible. When the experiment worked, but only at night, physicists scrambled to determine 
why (using Maxwell’s equations, of course). It was Oliver Heaviside, a mathematical physicist with strong 
engineering interests, who hypothesized that an invisible electromagnetic “mirror” surrounded the earth. 
What he meant was that at optical frequencies (and others as it turned out), the mirror was transparent, 
but at the frequencies Marconi used, it reflected electromagnetic radiation back to earth. He had predicted 
the existence of the ionosphere, a plasma that encompasses the earth at altitudes h; between 80 and 180 km 
that reacts to solar radiation: It becomes transparent at Marconi’s frequencies during the day, but becomes 
a mirror at night when solar radiation diminishes. The maximum distance along the earth’s surface that 


can be reached by a single ionospheric reflection is 2R cos , which ranges between 2,010 and 


1 

(5 + hy 
3,000 km when we substitute minimum and maximum ionospheric altitudes. This distance does not span 
the United States or cross the Atlantic; for transatlantic communication, at least two reflections would be 


2\/2Rh; +h,” 


required. The communication delay encountered with a single reflection in this channel is ; 
c 


which ranges between 6.8 and 10 ms, again a small time interval. 


6.7 Communication with Satellites® 


Global wireless communication relies on satellites. Here, ground stations transmit to orbiting satellites that 
amplify the signal and retransmit it back to earth. Satellites will move across the sky unless they are in 
geosynchronous orbits, where the time for one revolution about the equator exactly matches the earth’s 
rotation time of one day. TV satellites would require the homeowner to continually adjust his or her antenna 
if the satellite weren’t in geosynchronous orbit. Newton’s equations applied to orbiting bodies predict that 
the time T for one orbit is related to distance from the earth’s center R as 


3/GMT? 

R= te (6.20) 
where G is the gravitational constant and M the earth’s mass. Calculations yield R = 42200 km, which 
corresponds to an altitude of 35700 km. This altitude greatly exceeds that of the ionosphere, requiring 
satellite transmitters to use frequencies that pass through it. Of great importance in satellite communications 
is the transmission delay. The time for electromagnetic fields to propagate to a geosynchronous satellite and 
return is 0.24 s, a significant delay. 


Exercise 6.6 (Solution on p. 255.) 
In addition to delay, the propagation attenuation encountered in satellite communication far exceeds 
what occurs in ionospheric-mirror based communication. Calculate the attenuation incurred by 
radiation going to the satellite (one-way loss) with that encountered by Marconi (total going up 
and down). Note that the attenuation calculation in the ionospheric case, assuming the ionosphere 
acts like a perfect mirror, is not a straightforward application of the propagation loss formula 


(p. 201). 


’This content is available online at http: //cnx.org/content/m0539/2.10/. 
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6.8 Noise and Interference’ 


We have mentioned that communications are, to varying degrees, subject to interference and noise. It’s time 
to be more precise about what these quantities are and how they differ. 

Interference represents man-made signals. Telephone lines are subject to power-line interference (in 
the United States a distorted 60 Hz sinusoid). Cellular telephone channels are subject to adjacent-cell phone 
conversations using the same signal frequency. The problem with such interference is that it occupies the 
same frequency band as the desired communication signal, and has a similar structure. 


Exercise 6.7 (Solution on p. 255.) 
Suppose interference occupied a different frequency band; how would the receiver remove it? 


We use the notation i(t) to represent interference. Because interference has man-made structure, we can 
write an explicit expression for it that may contain some unknown aspects (how large it is, for example). 

Noise signals have little structure and arise from both human and natural sources. Satellite channels are 
subject to deep space noise arising from electromagnetic radiation pervasive in the galaxy. Thermal noise 
plagues all electronic circuits that contain resistors. Thus, in receiving small amplitude signals, receiver 
amplifiers will most certainly add noise as they boost the signal’s amplitude. All channels are subject to 
noise, and we need a way of describing such signals despite the fact we can’t write a formula for the noise 
signal like we can for interference. The most widely used noise model is white noise. It is defined entirely 
by its frequency-domain characteristics. 


White noise has constant power at all frequencies. 
At each frequency, the phase of the noise spectrum is totally uncertain: It can be any value in between 
0 and 27, and its value at any frequency is unrelated to the phase at any other frequency. 

e When noise signals arising from two different sources add, the resultant noise signal has a power equal 
to the sum of the component powers. 


Because of the emphasis here on frequency-domain power, we are led to define the power spectrum. 
Because of Parseval’s Theorem!°, we define the power spectrum P; (f) of a non-noise signal s(t) to be the 
magnitude-squared of its Fourier transform. 


Ps (f) = |S (f) |? (6.21) 


Integrating the power spectrum over any range of frequencies equals the power the signal contains in that 
band. Because signals must have negative frequency components that mirror positive frequency ones, we 
routinely calculate the power in a spectral band as the integral over positive frequencies multiplied by two. 


fo 
Power in[fi, fa] =2 f Ps (f)af (6.22) 
fi 
Using the notation n (t) to represent a noise signal’s waveform, we define noise in terms of its power spectrum. 
For white noise, the power spectrum equals the constant No, With this definition, the power in a frequency 
band equals No (f2 — fi). 

When we pass a signal through a linear, time-invariant system, the output’s spectrum equals the product 
(p. 119) of the system’s frequency response and the input’s spectrum. Thus, the power spectrum of the 

system’s output is given by 
P, (f) =|H (Ff) PPe (Ff) (6.23) 


This result applies to noise signals as well. When we pass white noise through a filter, the output is also a 


noise signal but with power spectrum |H (f) gow 


°This content is available online at http: //cnx.org/content/m0515/2.17/. 
10“Parseval’s Theorem,” (1) http: //cnx.org/content/m0047/latest/\#parseval 
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6.9 Channel Models" 


Both wireline and wireless channels share characteristics, allowing us to use a common model for how the 
channel affects transmitted signals. 


The transmitted signal is usually not filtered by the channel. 
The signal can be attenuated. 
The signal propagates through the channel at a speed equal to or less than the speed of light, which 
means that the channel delays the transmission. 
e The channel may introduce additive interference and/or noise. 


Letting a@ represent the attenuation introduced by the channel, the receiver’s input signal is related to the 
transmitted one by 
r(t) =aa(t—7)+i(t)+n(t) (6.24) 


This expression corresponds to the system model for the channel shown in Figure 6.4. In this book, we shall 
assume that the noise is white. 


x(t) r(t) al pel a tte atOn ome; Ue 


a laa To 
i(t 


Figure 6.4: The channel component of the fundamental model of communication (Figure 1.3) has the 
depicted form. The attenuation is due to propagation loss. Adding the interference and noise is justified 
by the linearity property of Maxwell’s equations. 


Exercise 6.8 (Solution on p. 255.) 
Is this model for the channel linear? 


As expected, the signal that emerges from the channel is corrupted, but does contain the transmitted signal. 
Communication system design begins with detailing the channel model, then developing the transmitter and 
receiver that best compensate for the channel’s corrupting behavior. We characterize the channel’s quality 
by the signal-to-interference ratio (STR) and the signal-to-noise ratio (SNR). The ratios are computed 
according to the relative power of each within the transmitted signal’s bandwidth. Assuming the 
signal x (t)’s spectrum spans the frequency interval [f;, f.,], these ratios can be expressed in terms of power 
spectra. 


20? [ P, (f) df 
SIR = 0 (6.25) 
2 P; (f) df 
fi 
2a? | P. (f) df 
= 0 
SNR = = A (6.26) 


In most cases, the interference and noise powers do not vary for a given receiver. Variations in signal-to- 
interference and signal-to-noise ratios arise from the attenuation because of transmitter-to-receiver distance 
variations. 


11This content is available online at http: //cnx.org/content/m0516/2.11/. 
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6.10 Baseband Communication” 


We use analog communication techniques for analog message signals, like music, speech, and television. 
Transmission and reception of analog signals using analog results in an inherently noisy received signal 
(assuming the channel adds noise, which it almost certainly does). 

The simplest form of analog communication is baseband communication. 


POINT OF INTEREST: We use analog communication techniques for analog message signals, like 
music, speech, and television. Transmission and reception of analog signals using analog results 
in an inherently noisy received signal (assuming the channel adds noise, which it almost certainly 
does). 


Here, the transmitted signal equals the message times a transmitter gain. 
x(t) = Gm(t) (6.27) 


An example, which is somewhat out of date, is the wireline telephone system. You don’t use baseband 
communication in wireless systems simply because low-frequency signals do not radiate well. The receiver in 
a baseband system can’t do much more than filter the received signal to remove out-of-band noise (interference 
is small in wireline channels). Assuming the signal occupies a bandwidth of W Hz (the signal’s spectrum 
extends from zero to W), the receiver applies a lowpass filter having the same bandwidth, as shown in 
Figure 6.5. 


t A 
r(t) LPF mip 


Figure 6.5: The receiver for baseband communication systems is quite simple: a lowpass filter having 
the same bandwidth as the signal. 


We use the signal-to-noise ratio of the receiver’s output m (t) to evaluate any analog-message commu- 
nication system. Assume that the channel introduces an attenuation a@ and white noise of spectral height 


N 
= The filter does not affect the signal component—we assume its gain is unity—but does filter the 


noise, removing frequency components above W Hz. In the filter’s output, the received signal power equals 
a’?G? power (m) and the noise power NoW, which gives a signal-to-noise ratio of 


a? G? power (m) 


SNRbaseband rary NoW 


(6.28) 
The signal term power (m) will be proportional to the bandwidth W; thus, in baseband communication the 
signal-to-noise ratio varies only with transmitter gain and channel attenuation and noise level. 


6.11 Modulated Communication” 


Especially for wireless channels, like commercial radio and television, but also for wireline systems like cable 
television, an analog message signal must be modulated: The transmitted signal’s spectrum occurs at much 
higher frequencies than those occupied by the signal. 


POINT OF INTEREST: We use analog communication techniques for analog message signals, like 
music, speech, and television. Transmission and reception of analog signals using analog results 
in an inherently noisy received signal (assuming the channel adds noise, which it almost certainly 
does). 


12This content is available online at http: //cnx.org/content/m0517/2.19/. 
13This content is available online at http: //cnx.org/content/m0518/2.26/. 
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(t) [ BPF | rt LPF | M(t) 
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Figure 6.6: The AM coherent receiver along with the spectra of key signals is shown for the case of a 
triangular-shaped signal spectrum. The dashed line indicates the white noise level. Note that the filters’ 
characteristics — cutoff frequency and center frequency for the bandpass filter — must be match to the 
modulation and message parameters. 


The key idea of modulation is to affect the amplitude, frequency or phase of what is known as the carrier 
sinusoid. Frequency modulation (FM) and less frequently used phase modulation (PM) are not discussed 
here; we focus on amplitude modulation (AM). The amplitude modulated message signal has the form 


x(t) = Ac(1 + m (E)) cos (27 fet) (6.29) 


where f, is the carrier frequency and A, the carrier amplitude. Also, the signal’s amplitude is assumed 
to be less than one: |m(t)| < 1. From our previous exposure to amplitude modulation (see the Fourier 
Transform example (Example 4.5)), we know that the transmitted signal’s spectrum occupies the frequency 
range [f. — W, fo + W], assuming the signal’s bandwidth is W Hz (see Figure 6.6). The carrier frequency 
is usually much larger than the signal’s highest frequency: (f. >> W), which means that the transmitter 
antenna and carrier frequency are chosen jointly during the design process. 

Ignoring the attenuation and noise introduced by the channel for the moment, reception of an amplitude 
modulated signal is quite easy (see Problem 4.20). The so-called coherent receiver multiplies the input 
signal by a sinusoid and lowpass-filters the result (Figure 6.6). 


in (t) = LPF [2 (t) cos (21 fet) 


6.30 
= LPF [A,(1 + m (t))cos* (27 fct)] ay) 
Because of our trigonometric identities, we know that 
1 
cos? (27 fet) = 5 (1 + cos (272 fct)) (6.31) 


At this point, the message signal is multiplied by a constant and a sinusoid at twice the carrier frequency. 
Multiplication by the constant term returns the message signal to baseband (where we want it to be!) while 
multiplication by the double-frequency term yields a very high frequency signal. The lowpass filter removes 
this high-frequency signal, leaving only the baseband signal. Thus, the received signal is 


Of “(1 +m (t)) (6.32) 


3) 


Exercise 6.9 (Solution on p. 256.) 
This derivation relies solely on the time domain; derive the same result in the frequency domain. 
You won't need the trigonometric identity with this approach. 
Because it is so easy to remove the constant term by electrical means—we insert a capacitor in series with 
the receiver’s output—we typically ignore it and concentrate on the signal portion of the receiver’s output 
when calculating signal-to-noise ratio. 
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6.12 Signal-to-Noise Ratio of an Amplitude-Modulated Signal” 


When we consider the much more realistic situation when we have a channel that introduces attenuation 
and noise, we can make use of the just-described receiver’s linear nature to directly derive the receiver’s 
output. The attenuation affects the output in the same way as the transmitted signal: It scales the output 
signal by the same amount. The white noise, on the other hand, should be filtered from the received signal 
before demodulation. We must thus insert a bandpass filter having bandwidth 2W and center frequency f-: 
This filter has no effect on the received signal-related component, but does remove out-of-band noise power. 
As shown in the triangular-shaped signal spectrum (Figure 6.6), we apply coherent receiver to this filtered 
signal, with the result that the demodulated output contains noise that cannot be removed: It lies in the 
same spectral band as the signal. 

As we derive the signal-to-noise ratio in the demodulated signal, let’s also calculate the signal-to-noise 
ratio of the bandpass filter’s output 7 (t). The signal component of 7 (t) equals ~A.m (t) cos (27 fet). This 
signal’s Fourier transform equals 


aA. 


5 (M (f + fe) +.M (f — fe)) (6.33) 
making the power spectrum, 
AS 
<= (1M (f + fe)? +1M (f - fc) ?) (6.34) 
Exercise 6.10 (Solution on p. 256.) 


If you calculate the magnitude-squared of the first equation, you don’t obtain the second unless 
you make an assumption. What is it? 


2 2 
C 


Thus, the total signal-related power in 7 (t) is a power (m). The noise power equals the integral of the 


noise power spectrum; because the power spectrum is constant over the transmission band, this integral 
equals the noise amplitude No times the filter’s bandwidth 2W. The so-called received signal-to-noise 
ratio — the signal-to-noise ratio after the de rigeur front-end bandpass filter and before demodulation — 
equals 

a? A.” power (m) 


SNR, = 
4NoW 


(6.35) 


aA.m (t) a A.” power (m) 


The demodulated signal m (t) = 
To determine the noise power, we must understand how the coherent demodulator affects the bandpass 
noise found in 7 (t). Because we are concerned with noise, we must deal with the power spectrum since we 
don’t have the Fourier transform available to us. Letting P(f) denote the power spectrum of # (t)’s noise 
component, the power spectrum after multiplication by the carrier has the form 


Pipes GF ge) 
4 


The delay and advance in frequency indicated here results in two spectral noise bands falling in the low- 
frequency region of lowpass filter’s passband. Thus, the total noise power in this filter’s output equals 


+ Nout (t). Clearly, the signal power equals 


(6.36) 


N 
(2 . ho -W-2- <) a ae The signal-to-noise ratio of the receiver’s output thus equals 
SNRq = a? A.” power (m) 
2NoW (6.37) 
= 2SNR,,. 


Let’s break down the components of this signal-to-noise ratio to better appreciate how the channel and 
the transmitter parameters affect communications performance. Better performance, as measured by the 
SNR, occurs as it increases. 


14This content is available online at http: //cnx.org/content/m0541/2.18/. 
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More transmitter power — increasing A, — increases the signal-to-noise ratio proportionally. 

The carrier frequency f. has no effect on SNR, but we have assumed that (f. >> W). 

The signal bandwidth W enters the signal-to-noise expression in two places: implicitly through the 
signal power and explicitly in the expression’s denominator. If the signal spectrum had a constant 
amplitude as we increased the bandwidth, signal power would increase proportionally. On the other 
hand, our transmitter enforced the criterion that signal amplitude was constant (Section 6.7). Signal 
amplitude essentially equals the integral of the magnitude of the signal’s spectrum. 


NOTE: This result isn’t exact, but we do know that m (0) = f° M (f) df. 


Enforcing the signal amplitude specification means that as the signal’s bandwidth increases we must de- 
crease the spectral amplitude, with the result that the signal power remains constant. Thus, increasing 
signal bandwidth does indeed decrease the signal-to-noise ratio of the receiver’s output. 

e Increasing channel attenuation — moving the receiver farther from the transmitter — decreases the 
signal-to-noise ratio as the square. Thus, signal-to-noise ratio decreases as distance-squared between 
transmitter and receiver. 

e Noise added by the channel adversely affects the signal-to-noise ratio. 


In summary, amplitude modulation provides an effective means for sending a bandlimited signal from one 
place to another. For wireline channels, using baseband or amplitude modulation makes little difference in 
terms of signal-to-noise ratio. For wireless channels, amplitude modulation is the only alternative. The one 
AM parameter that does not affect signal-to-noise ratio is the carrier frequency f-: We can choose any value 
we want so long as the transmitter and receiver use the same value. However, suppose someone else wants to 
use AM and chooses the same carrier frequency. The two resulting transmissions will add, and both receivers 
will produce the sum of the two signals. What we clearly need to do is talk to the other party, and agree to 
use separate carrier frequencies. As more and more users wish to use radio, we need a forum for agreeing on 
carrier frequencies and on signal bandwidth. On earth, this forum is the government. In the United States, 
the Federal Communications Commission (FCC) strictly controls the use of the electromagnetic spectrum 
for communications. Separate frequency bands are allocated for commercial AM, FM, cellular telephone 
(the analog version of which is AM), short wave (also AM), and satellite communications. 


Exercise 6.11 (Solution on p. 256.) 
Suppose all users agree to use the same signal bandwidth. How closely can the carrier frequencies 
be while avoiding communications crosstalk? What is the signal bandwidth for commercial AM? 
How does this bandwidth compare to the speech bandwidth? 


6.13 Digital Communication” 


Effective, error-free transmission of a sequence of bits—a bit stream {b (0) ,b(1),...}-is the goal here. We 
found that analog schemes, as represented by amplitude modulation, always yield a received signal containing 
noise as well as the message signal when the channel adds noise. Digital communication schemes are very 
different. Once we decide how to represent bits by analog signals that can be transmitted over wireline 
(like a computer network) or wireless (like digital cellular telephone) channels, we will then develop a way 
of tacking on communication bits to the message bits that will reduce channel-induced errors greatly. In 
theory, digital communication errors can be zero, even though the channel adds noise! 

We represent a bit by associating one of two specific analog signals with the bit’s value. Thus, if b(n) = 0, 
we transmit the signal so (t); if b(n) = 1, send sj (t). These two signals comprise the signal set for digital 
communication and are designed with the channel and bit stream in mind. In virtually every case, these 
signals have a finite duration T common to both signals; this duration is known as the bit interval. 
Exactly what signals we use ultimately affects how well the bits can be received. Interestingly, baseband 
and modulated signal sets can yield the same performance. Other considerations determine how signal set 
choice affects digital communication performance. 


15This content is available online at http: //cnx.org/content/m0519/2.10/. 
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So(t) $1 (t) 


Figure 6.7 
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Figure 6.8: The upper plot shows how a baseband signal set for transmitting the bit sequence 0110. 
The lower one shows an amplitude-modulated variant suitable for wireless channels. 


Exercise 6.12 (Solution on p. 256.) 
What is the expression for the signal arising from a digital transmitter sending the bit stream b(n), 
n={...,—-1,0,1,...} using the signal set so (t), 51 (¢), each signal of which has duration T? 


6.14 Binary Phase Shift Keying” 


A commonly used example of a signal set consists of pulses that are negatives of each other (Figure 6.7). 
so(t) = Apr (t) 
81 (t) = —Apr (t) 


Here, we have a baseband signal set suitable for wireline transmission. The entire bit stream b(n) is 
represented by a sequence of these signals. Mathematically, the transmitted signal has the form 


x(t) = S> (-1)" Apr (t — nT) (6.39) 


n 


(6.38) 


and graphically Figure 6.8 shows what a typical transmitted signal might be. 

This way of representing a bit stream—changing the bit changes the sign of the transmitted signal—is 
known as binary phase shift keying and abbreviated BPSK. The name comes from concisely expressing 
this popular way of communicating digital information. The word “binary” is clear enough (one binary-valued 
quantity is transmitted during a bit interval). Changing the sign of sinusoid amounts to changing—shifting— 
the phase by z (although we don’t have a sinusoid yet). The word “keying” reflects back to the first electrical 
communication system, which happened to be digital as well: the telegraph. 


16This content is available online at http: //cnx.org/content/m10280/2.14/. 
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The datarate R of a digital communication system is how frequently an information bit is transmitted. 
1 
In this example it equals the reciprocal of the bit interval: R = i Thus, for a 1 Mbps (megabit per second) 


transmission, we must have T' = 1 ps. 

The choice of signals to represent bit values is arbitrary to some degree. Clearly, we do not want to 
choose signal set members to be the same; we couldn’t distinguish bits if we did so. We could also have 
made the negative-amplitude pulse represent a 0 and the positive one a 1. This choice is indeed arbitrary 
and will have no effect on performance assuming the receiver knows which signal represents which bit. As 
in all communication systems, we design transmitter and receiver together. 

A simple signal set for both wireless and wireline channels amounts to amplitude modulating a baseband 
signal set (more appropriate for a wireline channel) by a carrier having a frequency harmonic with the bit 
interval. 


27k 
so(t) = Apr (#) sin (=) 
ae (6.40) 
So(t) $4 (t) 
A A 
T 
t t 
T 
Figure 6.9 
Exercise 6.13 (Solution on p. 256.) 


What is the value of k in this example? 


This signal set is also known as a BPSK signal set. We’ll show later that indeed both signal sets provide 
identical performance levels when the signal-to-noise ratios are equal. 


Exercise 6.14 (Solution on p. 256.) 
Write a formula, in the style of the baseband signal set, for the transmitted signal as shown in the 
plot of the baseband signal set!’ that emerges when we use this modulated signal. 


What is the transmission bandwidth of these signal sets? We need only consider the baseband version 
as the second is an amplitude-modulated version of the first. The bandwidth is determined by the bit 
sequence. If the bit sequence is constant—always 0 or always 1—the transmitted signal is a constant, which 
has zero bandwidth. The worst-case—bandwidth consuming—bit sequence is the alternating one shown in 
Figure 6.10. In this case, the transmitted signal is a square wave having a period of 2T. 


Figure 6.10: Here we show the transmitted waveform corresponding to an alternating bit sequence. 


17«Sienal Sets,” Figure 2 http://cnx.org/content/m0542/latest/\#£ig1001 
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From our work in Fourier series, we know that this signal’s spectrum contains odd-harmonics of the 
fundamental, which here equals = Thus, strictly speaking, the signal’s bandwidth is infinite. In practical 
terms, we use the 90%-power bandwidth to assess the effective range of frequencies consumed by the signal. 
The first and third harmonics contain that fraction of the total power, meaning that the effective bandwidth 
of our baseband signal is a or, expressing this quantity in terms of the datarate, on Thus, a digital 


communications signal requires more bandwidth than the datarate: a 1 Mbps baseband system requires a 
bandwidth of at least 1.5 MHz. Listen carefully when someone describes the transmission bandwidth of 
digital communication systems: Did they say “megabits” or “megahertz?” 


Exercise 6.15 (Solution on p. 256.) 
Show that indeed the first and third harmonics contain 90% of the transmitted power. If the receiver 


uses a front-end filter of bandwidth rT what is the total harmonic distortion of the received signal? 


Exercise 6.16 (Solution on p. 256.) 
What is the 90% transmission bandwidth of the modulated signal set? 


6.15 Frequency Shift Keying” 
In frequency-shift keying(FSK), the bit affects the frequency of a carrier sinusoid. 


80 (t) = Apr (t) sin (27 fot) 


81 (t) = Apr (t) sin (27 ft) (6.41) 


Figure 6.11 


The frequencies fo, f; are usually harmonically related to the bit interval. In the depicted example, 
3 
fo = = and f; = —. As can be seen from the transmitted signal for our example bit stream (Figure 6.12), 


the transitions at bit interval boundaries are smoother than those of BPSK. 
To determine the bandwidth required by this signal set, we again consider the alternating bit stream. 
Think of it as two signals added together: The first comprised of the signal so (t), the zero signal, so (¢), 


Figure 6.12: This plot shows the FSK waveform for same bitstream used in the BPSK example (Fig- 
ure 6.8). 
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Figure 6.13: The depicted decomposition of the FSK-modulated alternating bit stream into its fre- 
quency components simplifies the calculation of its bandwidth. 


zero, etc., and the second having the same structure but interleaved with the first and containing s; (t) 
(Figure 6.13). 

Each component can be thought of as a fixed-frequency sinusoid multiplied by a square wave of period 
2T that alternates between one and zero. This baseband square wave has the same Fourier spectrum as 
our BPSK example, but with the addition of the constant term co. This quantity’s presence changes the 
number of Fourier series terms required for the 90% bandwidth: Now we need only include the zero and first 


1 1 1 
harmonics to achieve it. The bandwidth thus equals, with fo < fi, fi 4 ( fo ) =fi-fot+t=. 


2T 2T T 
If the two frequencies are harmonics of the bit-interval duration, fo = a and f; = ra with ki > ko, the 


ky —ko +1 
bandwidth equals SEs, If the difference between harmonic numbers is 1, then the FSK bandwidth 
is smaller than the BPSK bandwidth. If the difference is 2, the bandwidths are equal and larger differences 
produce a transmission bandwidth larger than that resulting from using a BPSK signal set. 


6.16 Digital Communication Receivers” 


The receiver interested in the transmitted bit stream must perform two tasks when received waveform r (t) 
begins. 


e It must determine when bit boundaries occur: The receiver needs to synchronize with the transmitted 
signal. Because transmitter and receiver are designed in concert, both use the same value for the bit 
interval T. Synchronization can occur because the transmitter begins sending with a reference bit 
sequence, known as the preamble. This reference bit sequence is usually the alternating sequence 
as shown in the square wave example”? and in the FSK example (Figure 6.13). The receiver knows 
what the preamble bit sequence is and uses it to determine when bit boundaries occur. This procedure 
amounts to what in digital hardware as self-clocking signaling: The receiver of a bit stream must 
derive the clock — when bit boundaries occur — from its input signal. Because the receiver usually 
does not determine which bit was sent until synchronization occurs, it does not know when during the 
preamble it obtained synchronization. The transmitter signals the end of the preamble by switching 
to a second bit sequence. The second preamble phase informs the receiver that data bits are about to 
come and that the preamble is almost over. 

e Once synchronized and data bits are transmitted, the receiver must then determine every T' seconds 
what bit was transmitted during the previous bit interval. We focus on this aspect of the digital 
receiver because this strategy is also used in synchronization. 


18This content is available online at http: //cnx.org/content/m0545/2.12/. 
19This content is available online at http: //cnx.org/content/m0520/2.18/. 
20“Tyansmission Bandwidth,” Figure 1 http://cnx.org/content/m0544/latest/\#fig1003 
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The receiver for digital communication is known as a matched filter. 
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Figure 6.14: The optimal receiver structure for digital communication faced with additive white noise 
channels is the depicted matched filter. 


This receiver multiplies the received signal by each of the possible members of the transmitter signal set, 
integrates the product over the bit interval, and compares the results. Whichever path through the receiver 
yields the largest value corresponds to the receiver’s decision as to what bit was sent during the previous bit 
interval. For the next bit interval, the multiplication and integration begins again, with the next bit decision 
made at the end of the bit interval. Mathematically, the received value of b(n), which we label b(n), is given 
by 

(n+1)T 

b(n) = arg max; / r (t) 5; (t) dt (6.42) 

nT 


You may not have seen the arg max, notation before. max; {-} yields the maximum value of its argument 
with respect to the index 7. arg max, equals the value of the index that yields the maximum. Note that the 
precise numerical value of the integrator’s output does not matter; what does matter is its value relative to 
the other integrator’s output. 

Let’s assume a perfect channel for the moment: The received signal equals the transmitted one. If bit 0 
were sent using the baseband BPSK signal set, the integrator outputs would be 


(n+1)T 
/ r (t) so (t) dt = A?T 
yeu (6.43) 


(n+1)T 
/ r(t)s, (t)dt = -—A’T 
nT 


If bit 1 were sent, 


(n+1)T 
/ r (t) so (t) dt = —A’T 


T 
(mtr (6.44) 
/ r (t) sy (t) dt = A°T 

nT 
Exercise 6.17 (Solution on p. 256.) 
Can you develop a receiver for BPSK signal sets that requires only one multiplier-integrator com- 
bination? 
Exercise 6.18 (Solution on p. 256.) 


What is the corresponding result when the amplitude-modulated BPSK signal set is used? 


Clearly, this receiver would always choose the bit correctly. Channel attenuation would not affect this 
correctness; it would only make the values smaller, but all that matters is which is largest. 
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6.17 Digital Communication in the Presence of Noise” 


When we incorporate additive noise into our channel model, so that r (t) = as, (t) +n (t), errors can creep in. 
If the transmitter sent bit 0 using a BPSK signal set, the integrators’ outputs in the matched filter receiver 
(Figure 6.14) would be 


(n+1)T (n+1)T 
/ r(t)so(t)dt = aA?T + “ n(t) 89 (t) dt 


T T 
(n+1)T (n+1)T (6.45) 
/ r (t) 81 (t) dt = —-aA?T + / n(t) 51 (t) dt 
nT nT 


The quantities containing noise terms cause errors in the receiver’s decision-making process. Because they 
involve noise, the values of these integrals are random quantities drawn from some probability distribution 
that vary erratically from bit interval to bit interval. Because the noise has zero average value and has an 
equal amount of power in all frequency bands, the values of the integrals will hover about zero. What is 
important is how much they vary. If the noise is such that its integral term is more negative than aA?T, 
then the receiver will make an error, deciding that the transmitted zero-valued bit was indeed a one. The 
probability that this situation occurs depends on three factors: 


e Signal Set Choice — The difference between the signal-dependent terms in the integrators’ outputs 
(equations (6.45)) defines how large the noise term must be for an incorrect receiver decision to result. 
What affects the probability of such errors occurring is the energy in the difference of the received 
signals in comparison to the noise term’s variability. The signal-difference energy equals 


T 
if (<1 (#) — 89 (2))?at 


For our BPSK baseband signal set, the difference-signal-energy term is 4a? A*T?. 

e Variability of the Noise Term — We quantify variability by the spectral height of the white noise 
No added by the channel. 

e Probability Distribution of the Noise Term — The value of the noise terms relative to the 
signal terms and the probability of their occurrence directly affect the likelihood that a receiver error 
will occur. For the white noise we have been considering, the underlying distributions are Gaussian. 
Deriving the following expression for the probability the receiver makes an error on any bit transmission 


is complicated but can be found in other modules: here?? and here?°. 
T 2 
_o{ {a =a! 
Pe 2No 


2a? A2T 
=Q —— for the BPSK case 
0 


(6.46) 


Here Q (-) is the integral Q (x) = Tz a pee e~°’/2dq, This integral has no closed form expression, but it 


can be accurately computed. As Figure 6.15 illustrates, Q (-) is a decreasing, very nonlinear function. 
The term A?T equals the energy expended by the transmitter in sending the bit; we label this term E,. We 
arrive at a concise expression for the probability the matched filter receiver makes a bit-reception error when 
the BPSK signal set is used. 


2a? Ey 


Pe=Q No (6.47) 


21 This content is available online at http: //cnx.org/content/m0546/2.14/. 
22 "Detection of Signals in Noise" http: //cnx.org/content/m16253/latest/ 
23"Continuous-Time Detection Theory" http://cnx.org/content/m11406/latest/ 
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Figure 6.15: The function Q(z) is plotted in semilogarithmic coordinates. Note that it decreases very 
rapidly for small increases in its arguments. For example, when x increases from 4 to 5, Q (x) decreases 
by a factor of 100. 
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Figure 6.16: The probability that the matched-filter receiver makes an error on any bit transmission is 
plotted against the signal-to-noise ratio of the received signal. The upper curve shows the performance 
of the FSK signal set, the lower (and therefore better) one the BPSK signal set. 


2Ep 


‘ : . a 
Figure 6.16 shows how the receiver’s error rate varies with the signal-to-noise ratio 


0 
Exercise 6.19 (Solution on p. 256.) 
Derive the probability of error expression for the modulated BPSK signal set, and show that its 
performance identically equals that of the baseband BPSK signal set. 
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6.18 Digital Communication System Properties” 


Results from Section 6.17 reveal several properties about digital communication systems. 


e As the received signal becomes increasingly noisy, whether due to increased distance from the transmit- 
ter (smaller a) or to increased noise in the channel (larger No), the probability the receiver makes an 
error approaches 1/2. In such situations, the receiver performs only slightly better than the “receiver” 
that ignores what was transmitted and merely guesses what bit was transmitted. Consequently, it 
becomes almost impossible to communicate information when digital channels become noisy. 

e As the signal-to-noise ratio increases, performance gains-smaller probability of error pe — can be easily 
obtained. At a signal-to-noise ratio of 12 dB, the probability the receiver makes an error equals 107°. 
In words, one out of one hundred million bits will, on the average, be in error. 

e Once the signal-to-noise ratio exceeds about 5 dB, the error probability decreases dramatically. Adding 
1 dB improvement in signal-to-noise ratio can result in a factor of ten smaller pe. 

e Signal set choice can make a significant difference in performance. All BPSK signal sets, baseband or 
modulated, yield the same performance for the same bit energy. The BPSK signal set does perform 
much better than the FSK signal set once the signal-to-noise ratio exceeds about 5 dB. 


Exercise 6.20 (Solution on p. 256.) 
Derive the expression for the probability of error that would result if the FSK signal set were used. 


The matched-filter receiver provides impressive performance once adequate signal-to-noise ratios occur. You 
might wonder whether another receiver might be better. The answer is that the matched-filter receiver is 
optimal: No other receiver can provide a smaller probability of error than the matched filter 
regardless of the SNR. Furthermore, no signal set can provide better performance than the BPSK signal 
set, where the signal representing a bit is the negative of the signal representing the other bit. The reason 
for this result rests in the dependence of probability of error p. on the difference between the noise-free 
integrator outputs: For a given Ey, no other signal set provides a greater difference. 

How small should the error probability be? Out of N transmitted bits, on the average Np, bits will be 
received in error. Do note the phrase “on the average” here: Errors occur randomly because of the noise 
introduced by the channel, and we can only predict the probability of occurrence. Since bits are transmitted 
at a rate R, errors occur at an average frequency of Rp,. Suppose the error probability is an impressively 
small number like 10~°. Data on a computer network like Ethernet is transmitted at a rate R = 100Mbps, 
which means that errors would occur at a rate of roughly 100 per second. This error rate is very high, 
requiring a much smaller pe to achieve a more acceptable average occurrence rate for errors occurring. 
Because Ethernet is a wireline channel, which means the channel noise is small and the attenuation low, 
obtaining very small error probabilities is not difficult. We do have some tricks up our sleeves, however, that 
can essentially reduce the error rate to zero without resorting to expending a large amount of energy at 
the transmitter. We need to understand digital channels and what implications Shannon’s Noisy Channel 
Coding Theorem has for them. 


6.19 Digital Channels” 


Let’s review how digital communication systems work within the Fundamental Model of Communication 
(Figure 1.3). As depicted in Figure 6.17, the message is a single bit. The analog components of a digital 
transmission/reception system can be lumped into a single system known as the digital channel. 

Digital channels are described by transition diagrams, which indicate the output alphabet symbols 
that result for each possible transmitted symbol and the probabilities of the various reception possibilities. 
The probabilities on transitions coming from the same symbol must sum to one. For the matched-filter 
receiver and the signal sets we have seen, the depicted transition diagram, known as a binary symmetric 
channel, captures how transmitted bits are received. The probability of error pe is the sole parameter of 
the digital channel, and it encapsulates signal set choice, channel properties, and the matched-filter receiver. 
With this simple but entirely accurate model, we can concentrate on how bits are received. 


24This content is available online at http: //cnx.org/content/m10282/2.9/. 
25This content is available online at http: //cnx.org/content/m0102/2.14/. 
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Figure 6.17: The steps in transmitting digital information are shown in the upper system, the Funda- 
mental Model of Communication. The symbolic-valued signal s (m) forms the message, and it is encoded 
into a bit sequence b(n). The indices differ because more than one bit/symbol is usually required to 
represent the message by a bitstream. Each bit is represented by an analog signal, transmitted through 
the (unfriendly) channel, and received by a matched-filter receiver. From the received bitstream b(n) the 
received symbolic-valued signal $(m) is derived. The lower block diagram shows an equivalent system 
wherein the analog portions are combined and modeled by a transition diagram, which shows how each 
transmitted bit could be received. For example, transmitting a O results in the reception of a 1 with 
probability pe (an error) or a 0 with probability 1 — pe (no error). 


6.20 Entropy”® 


Communication theory has been formulated best for symbolic-valued signals. Claude Shannon?’ published 
in 1948 The Mathematical Theory of Communication, which became the cornerstone of digital communi- 
cation. He showed the power of probabilistic models for symbolic-valued signals, which allowed him to 
quantify the information present in a signal. In the simplest signal model, each symbol can occur at index 
n with a probability Pr [a,], k = {1,...,&}. What this model says is that for each signal value a K-sided 
coin is flipped (note that the coin need not be fair). For this model to make sense, the probabilities must be 
numbers between zero and one and must sum to one. 


0 < Pr [ax] <1 (6.48) 
K 

S$ Pr lax] =1 (6.49) 
k=1 


This coin-flipping model assumes that symbols occur without regard to what preceding or succeeding symbols 
were, a false assumption for typed text. Despite this probabilistic model’s over-simplicity, the ideas we 
develop here also work when more accurate, but still probabilistic, models are used. The key quantity that 
characterizes a symbolic-valued signal is the entropy of its alphabet. 


H (A) = —S~ Pr [ag] logy Pr [ax] (6.50) 
k 


Because we use the base-2 logarithm, entropy has units of bits. For this definition to make sense, we must 
take special note of symbols having probability zero of occurring. A zero-probability symbol never occurs; 
thus, we define Olog,0 = 0 so that such symbols do not affect the entropy. The maximum value attainable 
by an alphabet’s entropy occurs when the symbols are equally likely (Pr [az] = Pr|a;]). In this case, the 
entropy equals log,kK. The minimum value occurs when only one symbol occurs; it has probability one of 
occurring and the rest have probability zero. 


26 This content is available online at http: //cnx.org/content/m0070/2.13/. 
2"http: //www-gap.dcs.st-and.ac.uk/~history/Biographies/Shannon. html 
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Exercise 6.21 (Solution on p. 256.) 
Derive the maximum-entropy results, both the numeric aspect (entropy equals log,K) and the 
theoretical one (equally likely symbols maximize entropy). Derive the value of the minimum entropy 
alphabet. 

Example 6.1 

A four-symbol alphabet has the following probabilities. 


1 1 1 1 
Pr [ao] = 5 Pr [a,] = i Pr [ag] = Fi Pr [a3] = ; 


Note that these probabilities sum to one as they should. As $ =o), log, ( 
of this alphabet equals 


s) = —1. The entropy 


1, 1 Lh 1 1, 1 4 1 
9 082 \5 ea | r 9082 \ 3 rg 082\ 3 


=~ (50-45-2453) +5-3)] ey 


H(A) 


I 


2 
= 1.75 bits 


6.21 Source Coding Theorem”* 


The significance of an alphabet’s entropy rests in how we can represent it with a sequence of bits. Bit 
sequences form the “coin of the realm” in digital communications: they are the universal way of representing 
symbolic-valued signals. We convert back and forth between symbols to bit-sequences with what is known 
as a codebook: a table that associates symbols to bit sequences. In creating this table, we must be able to 
assign a unique bit sequence to each symbol so that we can go between symbol and bit sequences without 
error. 


NOTE: You may be conjuring the notion of hiding information from others when we use the 
name codebook for the symbol-to-bit-sequence table. There is no relation to cryptology, which 
comprises mathematically provable methods of securing information. The codebook terminology 
was developed during the beginnings of information theory just after World War II. 


As we shall explore in some detail elsewhere, digital communication is the transmission of symbolic-valued 
signals from one place to another. When faced with the problem, for example, of sending a file across the 
Internet, we must first represent each character by a bit sequence. Because we want to send the file quickly, 
we want to use as few bits as possible. However, we don’t want to use so few bits that the receiver cannot 
determine what each character was from the bit sequence. For example, we could use one bit for every 
character: File transmission would be fast but useless because the codebook creates errors. Shannon proved 
in his monumental work what we call today the Source Coding Theorem. Let B (a,) denote the number 
of bits used to represent the symbol az. The average number of bits B (A) required to represent the entire 
alphabet equals Be B (ax) Pr [ax]. The Source Coding Theorem states that the average number 
of bits needed to accurately represent the alphabet need only to satisfy 


H(A) <B(A)<H(A)4+1 (6.52) 


Thus, the alphabet’s entropy specifies to within one bit how many bits on the average need to be used to 
send the alphabet. The smaller an alphabet’s entropy, the fewer bits required for digital transmission of files 
expressed in that alphabet. 


28This content is available online at http: //cnx.org/content/m0091/2.13/. 
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Example 6.2 
A four-symbol alphabet has the following probabilities. 
1 1 1 1 

Pr [ao] = 5 Pr [a1] = ry Pr [a2] = 3 Pr [a3] = 3 
and an entropy of 1.75 bits (Example 6.1). Let’s see if we can find a codebook for this four-letter 
alphabet that satisfies the Source Coding Theorem. The simplest code to try is known as the 
simple binary code: convert the symbol’s index into a binary number and use the same number 
of bits for each symbol by including leading zeros where necessary. 


(do o> 00) (a1 <> 01) (a2 oO 10) (a3 oO 11) (6.53) 


Whenever the number of symbols in the alphabet is a power of two (as in this case), the average 
number of bits B(A) equals log,K, which equals 2 in this case. Because the entropy equals 1.75 
bits, the simple binary code indeed satisfies the Source Coding Theorem—we are within one bit 
of the entropy limit—but you might wonder if you can do better. If we chose a codebook with 
differing number of bits for the symbols, a smaller average number of bits could indeed be obtained. 
The idea is to use shorter bit sequences for the symbols that occur more often. One codebook like 
this is 
(a9 @ 0) (a1 4 10) (a2 4 110) (ag 4 111) (6.54) 


Now B(A) =1-5+2-44+3-§+3-§ =1.75. We can reach the entropy limit! The simple binary 
code is, in this case, less efficient than the unequal-length code. Using the efficient code, we can 
transmit the symbolic-valued signal having this alphabet 12.5% faster. Furthermore, we know that 
no more efficient codebook can be found because of Shannon’s Theorem. 


6.22 Compression and the Huffman Code” 


Shannon’s Source Coding Theorem (6.52) has additional applications in data compression. Here, we have 
a symbolic-valued signal source, like a computer file or an image, that we want to represent with as few 
bits as possible. Compression schemes that assign symbols to bit sequences are known as lossless if they 
obey the Source Coding Theorem; they are lossy if they use fewer bits than the alphabet’s entropy. Using 
a lossy compression scheme means that you cannot recover a symbolic-valued signal from its compressed 
version without incurring some error. You might be wondering why anyone would want to intentionally 
create errors, but lossy compression schemes are frequently used where the efficiency gained in representing 
the signal outweighs the significance of the errors. 

Shannon’s Source Coding Theorem states that symbolic-valued signals require on the average at least 
H (A) number of bits to represent each of its values, which are symbols drawn from the alphabet A. In the 
Section 6.21, we found that using a so-called fixed rate source coder, one that produces a fixed number of 
bits/symbol, may not be the most efficient way of encoding symbols into bits. What is not discussed there is 
a procedure for designing an efficient source coder: one guaranteed to produce the fewest bits/symbol on 
the average. That source coder is not unique, and one approach that does achieve that limit is the Huffman 
source coding algorithm. 


POINT OF INTEREST: In the early years of information theory, the race was on to be the first to find 
a provably maximally efficient source coding algorithm. The race was won by then MIT graduate 
student David Huffman®? in 1954, who worked on the problem as a project in his information theory 
course. We’re pretty sure he received an “A.” 


e Create a vertical table for the symbols, the best ordering being in decreasing order of probability. 


2°This content is available online at http: //cnx.org/content/m0092/2.19/. 
30nttps: //en.wikipedia. org/wiki/David_A._Huffman 


221 


e Form a binary tree to the right of the table. A binary tree always has two branches at each node. 
Build the tree by merging the two lowest probability symbols at each level, making the probability of 
the node equal to the sum of the merged nodes’ probabilities. If more than two nodes/symbols share 
the lowest probability at a given level, pick any two; your choice won’t affect B (A). 

e At each node, label each of the emanating branches with a binary number. The bit sequence obtained 
from passing from the tree’s root to the symbol is its Huffman code. 


Example 6.3 
The simple four-symbol alphabet used in the Entropy (Example 6.1) and Source Coding (Exam- 
ple 6.2) modules has a four-symbol alphabet with the following probabilities, 


1 1 il 1 
Pr [ao] = 5 Pr [a,] = Z Pr [a2] = ; Pr [a3] = 5 


and an entropy of 1.75 bits (Example 6.1). This alphabet has the Huffman coding tree shown in 
Figure 6.18. 


Symbol Probability Source Code 
ay 5 
ao i 
a3 4 
a4 4 


Figure 6.18: We form a Huffman code for a four-letter alphabet having the indicated probabilities of 
occurrence. The binary tree created by the algorithm extends to the right, with the root node (the one 
at which the tree begins) defining the codewords. The bit sequence obtained by traversing the tree from 
the root to the symbol defines that symbol’s binary code. 


The code thus obtained is not unique as we could have labeled the branches coming out of 
each node differently. The average number of bits required to represent this alphabet equals 
1.75 bits, which is the Shannon entropy limit for this source alphabet. If we had the symbolic- 
valued signal s(m) = {d2, a3, 41,4, @1,@2,...}, our Huffman code would produce the bitstream 
b(n) = 101100111010... 

If the alphabet probabilities were different, clearly a different tree, and therefore different code, 
could well result. Furthermore, we may not be able to achieve the entropy limit. If our symbols 
had the probabilities Pr [ai] = 5, Pr [a2] = $, Pr[a3] = 4, and Pr [a4] = 35, the average number 
of bits/symbol resulting from the Huffman coding algorithm would equal 1.75 bits. However, the 
entropy limit is 1.68 bits. The Huffman code does satisfy the Source Coding Theorem—its average 
length is within one bit of the alphabet’s entropy—but you might wonder if a better code existed. 
David Huffman showed mathematically that no other code could achieve a shorter average code 
than his. We can’t do better. 


Exercise 6.22 (Solution on p. 257.) 
Derive the Huffman code for this second set of probabilities, and verify the claimed average code 
length and alphabet entropy. 
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6.23 Subtleties of Coding” 


In the Huffman code, the bit sequences that represent individual symbols can have differing lengths so the 
bitstream index m does not increase in lock step with the symbol-valued signal’s index n. To capture how 
often bits must be transmitted to keep up with the source’s production of symbols, we can only compute 


averages. If our source code averages B (A) bits/symbol and symbols are produced at a rate R, the average 
bit rate equals B (A) R, and this quantity determines the bit interval duration T. 


Exercise 6.23 Ges (Solution on p. 257.) 
Calculate the relation between T and the average bit rate B (A) R. 


A subtlety of source coding is whether we need “commas” in the bitstream. When we use an unequal number 
of bits to represent symbols, how does the receiver determine when symbols begin and end? If you created a 
source code that required a separation marker in the bitstream between symbols, it would be very inefficient 
since you are essentially requiring an extra symbol in the transmission stream. 


NOTE: A good example of this need is the Morse Code: Between each letter, the telegrapher needs 
to insert a pause to inform the receiver when letter boundaries occur. 


As shown in this example (Example 6.3), no commas are placed in the bitstream, but you can unambiguously 
decode the sequence of symbols from the bitstream. Huffman showed that his (maximally efficient) code had 
the prefix property: No code for a symbol began another symbol’s code. Once you have the prefix property, 
the bitstream is partially self-synchronizing: Once the receiver knows where the bitstream starts, we can 
assign a unique and correct symbol sequence to the bitstream. 


Exercise 6.24 (Solution on p. 257.) 
Sketch an argument that prefix coding, whether derived from a Huffman code or not, will provide 
unique decoding when an unequal number of bits/symbol are used in the code. 


However, having a prefix code does not guarantee total synchronization: After hopping into the middle of a 
bitstream, can we always find the correct symbol boundaries? The self-synchronization issue does mitigate 
the use of efficient source coding algorithms. 


Exercise 6.25 (Solution on p. 257.) 
Show by example that a bitstream produced by a Huffman code is not necessarily self-synchronizing. 
Are fixed-length codes self synchronizing? 


Another issue is bit errors induced by the digital channel; if they occur (and they will), synchronization can 
easily be lost even if the receiver started “in synch” with the source. Despite the small probabilities of error 
offered by good signal set design and the matched filter, an infrequent error can devastate the ability to 
translate a bitstream into a symbolic signal. We need ways of reducing reception errors without demanding 
that p. be smaller. 


Example 6.4 

The first electrical communications system—the telegraph—was digital. When first deployed in 
1844, it communicated text over wireline connections using a binary code—the Morse code—to 
represent individual letters. To send a message from one place to another, telegraph operators 
would tap the message using a telegraph key to another operator, who would relay the message 
on to the next operator, presumably getting the message closer to its destination. In short, the 
telegraph relied on a network not unlike the basics of modern computer networks. To say it 
presaged modern communications would be an understatement. It was also far ahead of some 
needed technologies, namely the Source Coding Theorem. The Morse code, shown in Figure 6.19, 
was not a prefix code. To separate codes for each letter, Morse code required that a space—a 
pause—be inserted between each letter. In information theory, that space counts as another code 
letter, which means that the Morse code encoded text with a three-letter source code: dots, dashes 
and space. The resulting source code is not within a bit of entropy, and is grossly inefficient (about 
25%). Figure 6.19 shows a Huffman code for English text, which as we know is efficient. 


31 This content is available online at http: //cnx.org/content/m0093/2.16/. 
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% Morse Code | Huffman Code 
A 6.22 - 1011 
B 1.32 mae 010100 
Cc 3.11 =. 10101 
D 2.97 =. 01011 
E 10.53 | . 001 
F 1.68 ane 110001 
G | 1.65 --. 110000 
H 3.63 oad 11001 
I 6.14 7 1001 
J 0.06 — 01010111011 
K } 0.31 = 01010110 
L 3.07 wes 10100 
M | 2.48 -- 00011 
N | 5.73 -. 0100 
O | 6.06 --- 1000 
P 1.87 o. 00000 
Q 0.10 a 0101011100 
R | 5.87 -. 0111 
S 5.81 on 0110 
T | 7.68 - 1101 
Us| 2.27 ao 00010 
V_ | 0.70 wen 0101010 
W | 1.18 - 000011 
X 0.25 me 010101111 
Y | 1.07 =o 000010 
Z 0.06 --.. 0101011101011 


Figure 6.19: Morse and Huffman Codes for American-Roman Alphabet. The % column indicates the 
average probability (expressed in percent) of the letter occurring in English. The entropy H (A) of the 
this source is 4.14 bits. The average Morse codeword length is 2.5 symbols. Adding one more symbol for 
the letter separator and converting to bits yields an average codeword length of 5.56 bits. The average 
Huffman codeword length is 4.35 bits. 


6.24 Channel Coding” 


We can, to some extent, correct errors made by the receiver with only the error-filled bit stream emerging 
from the digital channel available to us. The idea is for the transmitter to send not only the symbol-derived 
bits emerging from the source coder but also additional bits derived from the coder’s bit stream. These 
additional bits, the error correcting bits, help the receiver determine if an error has occurred in the 
data bits (the important bits) or in the error-correction bits. Instead of the communication model shown 
previously (Figure 6.17), the transmitter inserts a channel coder before analog modulation, and the receiver 
the corresponding channel decoder (Figure 6.20). This block diagram shown there forms the Fundamental 
Model of Digital Communication. 

Shannon’s Noisy Channel Coding Theorem says that if the data aren’t transmitted too quickly, that error 
correction codes exist that can correct all the bit errors introduced by the channel. Unfortunately, Shannon 
did not demonstrate an error correcting code that would achieve this remarkable feat; in fact, no one has 
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Figure 6.20: To correct errors that occur in the digital channel, a channel coder and decoder are added 
to the communication system. Properly designed channel coding can greatly reduce the probability (from 
the uncoded value of p-) that a data bit b(n) is received incorrectly even when the probability of c (1) 
be received in error remains pe or becomes larger. This system forms the Fundamental Model of Digital 
Communication. 
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Figure 6.21: The upper portion depicts the result of directly modulating the bit stream b(n) into a 
transmitted signal x(t) using a baseband BPSK signal set. R is the datarate produced by the source 
coder. If that bit stream passes through a (3,1) channel coder to yield the bit stream c(l), the resulting 
transmitted signal requires a bit interval T three times smaller than the uncoded version. This reduction 
in the bit interval means that the transmitted energy/bit decreases by a factor of three, which results in 


an increased error probability in the receiver. 


found such a code. Shannon’s result proves it exists; seems like there is always more work to do. In any case, 
that should not prevent us from studying commonly used error correcting codes that not only find their way 
into all digital communication systems, but also into CDs and bar codes used on merchandise. 


6.25 Repetition Codes” 


Perhaps the simplest error correcting code is the repetition code. Here, the transmitter sends the data 
bit several times, an odd number of times in fact. Because the error probability p, is always less than s, we 
know that more of the bits should be correct rather than in error. Simple majority voting of the received 
bits (hence the reason for the odd number) determines the transmitted bit more accurately than sending 
it alone. For example, let’s consider the three-fold repetition code: for every bit b(n) emerging from the 
source coder, the channel coder produces three. Thus, the bit stream emerging from the channel coder c (1) 
has a data rate three times higher than that of the original bit stream b(n). The coding table (Table 1.1) 
illustrates when the majority-vote decoder can correct errors and when it can’t. 

Thus, if one bit of the three bits is received in error, the receiver can correct the error; if more than 
one error occurs, the channel decoder announces the bit is 1 instead of transmitted value of 0. Using this 
repetition code, the probability of b(n) 4 0 equals 3p,” (1 — pe) + pe. This probability of a decoding error 
is always less than pe, the uncoded value, so long as pe < $. 
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Received Codeword | Probability | Decoded Bit 
000 (1— pe)” 0 
001 pe(1— pe)? 0 
010 pe(1— pe)? 0 
011 De? (1 — De) 1 
100 Pe(1 — pe)” 0 
101 De? (1 — De) 1 
110 De? (1 — De) 1 
111 De? 1 


Table 6.1: In this example, the transmitter encodes 0 as 000. The channel creates an error 
(changing a 0 into a 1) with probability p.. The first column lists all possible received data words 
and the second the probability of each data word being received. The last column shows the 
results of the majority-vote decoder. When the decoder produces 0, it successfully corrected the 
errors introduced by the channel (if there were any; the top row corresponds to the case in which 
no errors occurred). The error probability of the decoders is the sum of the probabilities when 
the decoder produces 1. 


Exercise 6.26 (Solution on p. 257.) 
Demonstrate mathematically that this claim is indeed true. Is 3p.? (1 — pe) + pe® < pe? 


6.26 Block Channel Coding” 


Because of the higher datarate imposed by the channel coder, the probability of bit error occurring in the 
digital channel increases relative to the value obtained when no channel coding is used. The bit interval 
duration must be reduced by is in comparison to the no-channel-coding situation, which means the energy 
per bit E, goes down by the same amount. The bit interval must decrease by a factor of three if the 
transmitter is to keep up with the data stream, as illustrated in Figure 6.21. 


POINT OF INTEREST: It is unlikely that the transmitter’s power could be increased to compensate. 
Such is the sometimes-unfriendly nature of the real world. 


Because of this reduction, the error probability p,. of the digital channel goes up. The question thus becomes 
does channel coding really help: Is the effective error probability lower with channel coding even though the 
error probability for each transmitted bit is larger? The answer is no: Using a repetition code for channel 
coding cannot ultimately reduce the probability that a data bit is received in error. The ultimate reason is 
the repetition code’s inefficiency: transmitting one data bit for every three transmitted is too inefficient for 
the amount of error correction provided. 


Exercise 6.27 (Solution on p. 257.) 
Using MATLAB, calculate the probability a bit is received incorrectly with a three-fold repetition 
code. Show that when the energy per bit Ey is reduced by 1/3 that this probability is larger than 
the no-coding probability of error. 


The repetition code (p. 224) represents a special case of what is known as block channel coding. For every 
K bits that enter the block channel coder, it inserts an additional N — K error-correction bits to produce a 
block of N bits for transmission. We use the notation (N, K) to represent a given block code’s parameters. 
In the three-fold repetition code (p. 224), K = 1 and N = 3. A block code’s coding efficiency FE equals 
the ratio K/N, and quantifies the overhead introduced by channel coding. The rate at which bits must be 
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transmitted again changes: So-called data bits b(n) emerge from the source coder at an average rate B (A) 
and exit the channel at a rate 1/F higher. We represent the fact that the bits sent through the digital 
channel operate at a different rate by using the index / for the channel-coded bit stream c(l). Note that 
the blocking (framing) imposed by the channel coder does not correspond to symbol boundaries in the bit 
stream b(n), especially when we employ variable-length source codes. 

Does any error-correcting code reduce communication errors when real-world constraints are taken into 
account? The answer now is yes. To understand channel coding, we need to develop first a general framework 
for channel coding, and discover what it takes for a code to be maximally efficient: Correct as many errors 
as possible using the fewest error correction bits as possible (making the efficiency K’/N as large as possible). 


6.27 Error-Correcting Codes: Hamming Distance” 
So-called linear codes create error-correction bits by combining the data bits linearly. The phrase “linear 
combination” means here single-bit binary arithmetic. 

06@0=0 161=0 O0@1=1 160=1 

0-0=0 1-1l=1 0-1=0 1-0=0 


For example, let’s consider the specific (3,1) error correction code described by the following coding table 
and, more concisely, by the succeeding matrix expression. 


c(1) = b(1) 

c(2) = b(1) 

c(3) = b(1) 
c=Gb 

where 
1 c(1) 
G=|1)  e= Je] b= [oqa)| 

1 c(3) 


The length-K (in this simple example K = 1) block of data bits is represented by the vector b, and the 
length-N output block of the channel coder, known as a codeword, by c. The generator matrix G defines 
all block-oriented linear channel coders. 

As we consider other block codes, the simple idea of the decoder taking a majority vote of the received bits 
won’t generalize easily. We need a broader view that takes into account the distance between codewords. 
A length-N codeword means that the receiver must decide among the 2% possible data words to select 
which of the 2* codewords was actually transmitted. As shown in Figure 6.22, we can think of the data 
words geometrically. We define the Hamming distance between binary data words c; and cz, denoted by 
d(ci,¢2) to be the minimum number of bits that must be “flipped” to go from one word to the other. For 
example, the distance between codewords is 3 bits. In our table of binary arithmetic, we see that adding 
a 1 corresponds to flipping a bit. Furthermore, subtraction and addition are equivalent. We can express the 
Hamming distance as 


d (cy, c2) = sum ((c; @ c2)) (6.55) 


Exercise 6.28 (Solution on p. 258.) 
Show that adding the error vector col[1,0,...,0] to a codeword flips the codeword’s leading bit and 
leaves the rest unaffected. 
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Figure 6.22: Ina (3,1) repetition code, only two of the possible eight three-bit data blocks are codewords. 
We can represent these bit patterns geometrically with the axes being bit positions in the data block. In 
the left plot, the filled circles represent the codewords [0 0 0] and [1 1 1], the only possible codewords. 
The unfilled ones correspond to the transmission. The center plot shows that the distance between 
codewords is three. Because distance corresponds to flipping a bit, calculating the Hamming distance 
geometrically means following the axes rather than going “as the crow flies.” The right plot shows the 
data words that result when one error occurs as the codeword goes through the channel. The three data 
words are unit distance from the original codeword. Note that the received data word groups do not 
overlap, which means the code can correct all single-bit errors. 


The probability of one bit being flipped anywhere in a codeword is Np.(1 — as The number of errors 
the channel introduces equals the number of ones in e; the probability of any particular error vector decreases 
with the number of errors. 

To perform decoding when errors occur, we want to find the codeword (one of the filled circles in Fig- 
ure 6.22) that has the highest probability of occurring: the one closest to the one received. Note that if a data 
word lies a distance of one from two codewords, it is impossible to determine which codeword was actually 
sent. This criterion means that if any two codewords are two bits apart, then the code cannot correct the 
channel-induced error. Thus, to have a code that can correct all single-bit errors, codewords must 
have a minimum separation of three. Our repetition code has this property. 

Introducing code bits increases the probability that any bit arrives in error (because bit interval durations 
decrease). However, using a well-designed error-correcting code corrects bit reception errors. Do we win or 
lose by using an error-correcting code? The answer is that we can win if the code is well-designed. The (3,1) 
repetition code demonstrates that we can lose (Exercise 6.28). To develop good channel coding, we need to 
develop first a general framework for channel codes and discover what it takes for a code to be maximally 
efficient: Correct as many errors as possible using the fewest error correction bits as possible (making the 
efficiency K/N as large as possible.) We also need a systematic way of finding the codeword closest to any 
received data word. A much better code than our (3,1) repetition code is the following (7,4) code. 


e(1) =6(1) 
c(2) = b(2) 
c(3) = b(3) 
c(4) = b(4) 
c(5) = b(1) 6 b(2) © b(3) 
c(6) = (2) 6 b(3) & b(4) 
c(7) = (1) 66(2) © b(4) 
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where the generator matrix is 


1 0 0 0 
0 1 0 0 
001 0 
G=|0 00 1 
1 1 1 0 
0 LoL 
Ay at. Or od 


In this (7,4) code, 24 = 16 of the 2? = 128 possible blocks at the channel decoder correspond to error-free 
transmission and reception. 

Error correction amounts to searching for the codeword c closest to the received block ¢ in terms of the 
Hamming distance between the two. The error correction capability of a channel code is limited by how close 
together any two error-free blocks are. Bad codes would produce blocks close together, which would result 
in ambiguity when assigning a block of data bits to a received block. The quantity to examine, therefore, in 
designing code error correction codes is the minimum distance between codewords. 


dmin = min (d(qj,¢j)) , « AC; (6.56) 


To have a channel code that can correct all single-bit errors, dmin > 3. 


Exercise 6.29 (Solution on p. 258.) 
Suppose we want a channel code to have an error-correction capability of n bits. What must the 
minimum Hamming distance between codewords din be? 


How do we calculate the minimum distance between codewords? Because we have 2* codewords, the number 
of possible unique pairs equals 2*~'. (oe - 1), which can be a large number. Recall that our channel coding 
procedure is linear, with c = Gb. Therefore (c; @ c;) = G((b; © b;)). Because b; © b; always yields another 
block of data bits, we find that the difference between any two codewords is another codeword! Thus, to 
find dmin we need only compute the number of ones that comprise all non-zero codewords. Finding these 
codewords is easy once we examine the coder’s generator matrix. Note that the columns of G are codewords 
(why is this?), and that all codewords can be found by all possible pairwise sums of the columns. To find 
dmin, we need only count the number of bits in each column and sums of columns. For our example (7,4), 
G’s first column has three ones, the next one four, and the last two three. Considering sums of column pairs 
next, note that because the upper portion of G is an identity matrix, the corresponding upper portion of all 
column sums must have exactly two bits. Because the bottom portion of each column differs from the other 
columns in at least one place, the bottom portion of a sum of columns must have at least one bit. Triple 
sums will have at least three bits because the upper portion of G is an identity matrix. Thus, no sum of 
columns has fewer than three bits, which means that dj, = 3, and we have a channel coder that can correct 
all occurrences of one error within a received 7-bit block. 


6.28 Error-Correcting Codes: Channel Decoding” 


Because the idea of channel coding has merit (so long as the code is efficient), let’s develop a systematic 
procedure for performing channel decoding. One way of checking for errors is to try recreating the error 
correction bits from the data portion of the received block ¢. Using matrix notation, we make this calculation 
by multiplying the received block ¢ by the matrix H known as the parity check matrix. It is formed from 
the generator matrix G by taking the bottom, error-correction portion of G and attaching to it an identity 
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matrix. For our (7,4) code, 
1 1 1 0 1 0 0 
Sul) 10" ee De (6.57) 
1 101 0 0 1 


{Ss 
Lower portion of G Identity 


The parity check matrix thus has size (N — Kk) x N, and the result of multiplying this matrix with a received 

word is a length- (N — K) binary vector. If no digital channel errors occur—we receive a codeword so that 

€ = c — then Hc = 0. For example, the first column of G, (1,0,0,0,1,0,1), is a codeword. Simple 

calculations show that multiplying this vector by H results in a length-(V — kK’) zero-valued vector. 
Exercise 6.30 (Solution on p. 258.) 
Show that He = 0 for all the columns of G. In other words, show that HG =0 an (N-—K)x K 
matrix of zeroes. Does this property guarantee that all codewords also satisfy He = 0? 


When the received bits € do not form a codeword, H¢ does not equal zero, indicating the presence of one or 
more errors induced by the digital channel. Because the presence of an error can be mathematically written 
as C = (c Ge), with e a vector of binary values having a 1 in those positions where a bit error occurred 
(Table 6.3). 


e He 
1000000 | 101 
0100000 | 111 
0010000 | 110 
0001000 | O11 
0000100 | 100 
0000010 | 010 
0000001 | 001 


Table 6.3: The single-bit error patterns in the left column yield the decoding sequences shown in the right column. 


Exercise 6.31 (Solution on p. 258.) 
Show that adding the error vector (1,0,... ,0)" to a codeword flips the codeword’s leading bit and 
leaves the rest unaffected. 


Consequently, HC = H (ec Ge) = He. Because the result of the product is a length- (N — K) vector of binary 
values, we can have 2N~-* — 1 non-zero values that correspond to non-zero error patterns e. To perform our 
channel decoding, 


1. compute (conceptually at least) Hc; 

2. if this result is zero, no detectable or correctable error occurred; 

3. if non-zero, consult a table of length-(V — kK) binary vectors to associate them with the minimal error 
pattern that could have resulted in the non-zero result; then 

4. add the error vector thus obtained to the received vector ¢ to correct the error (because (¢c @ e G e) = c). 

5. Select the data bits from the corrected word to produce the received bit sequence b(n). 


The phrase minimal in the third item raises the point that a double (or triple or quadruple ...) error 
occurring during the transmission/reception of one codeword can create the same received word as a single- 
bit error or no error in another codeword. For example, (1, 0,0, 0,1, 0, 1\" and (0,1,0,0,1,1, ie are both 
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N K E 
1 | 0.33 
7 4 | 0.57 
15 11 | 0.73 
31 26 | 0.84 
63 57 | 0.90 
127 | 120 | 0.94 
Table 6.4: The codeword lengths N and data-block lengths K that form a perfect single-bit error correcting code 


are shown, along with the coding efficiency found by EF = K/N. 


codewords in the example (7,4) code. The second results when the first one experiences three bit errors 
(first, second, and sixth bits). Such an error pattern cannot be detected by our coding strategy, but such 
multiple error patterns are very unlikely to occur. Our receiver uses the principle of maximum probability: 
An error-free transmission is much more likely than one with three errors if the bit-error probability p. is 
small enough. 


Exercise 6.32 (Solution on p. 258.) 
How small must pe be so that a single-bit error is more likely to occur than a triple-bit error? 


6.29 Error-Correcting Codes: Hamming Codes” 


For the (7,4) example, we have 2V~* — 1 = 7 error patterns that can be corrected. We start with single-bit 


error patterns, and multiply them by the parity check matrix. If we obtain unique answers, we are done; if 
two or more error patterns yield the same result, we can try double-bit error patterns. In our case, single-bit 
error patterns give a unique result. 

This corresponds to our decoding table: We associate the parity check matrix multiplication result with 
the error pattern and add this to the received word. If more than one error occurs (unlikely though it may 
be), this “error correction” strategy usually makes the error worse in the sense that more bits are changed 
from what was transmitted. 

As with the repetition code, we must question whether our (7,4) code’s error correction capability com- 
pensates for the increased error probability due to the necessitated reduction in bit energy. Figure 6.23 shows 
that if the signal-to-noise ratio is large enough channel coding yields a smaller error probability. Because the 
bit stream emerging from the source decoder is segmented into four-bit blocks, the fair way of comparing 
coded and uncoded transmission is to compute the probability of block error: the probability that any bit 
in a block remains in error despite error correction and regardless of whether the error occurs in the data 
or in coding buts. Clearly, our (7,4) channel code does yield smaller error rates, and is worth the additional 
systems required to make it work. 

Note that our (7,4) code has the length and number of data bits that perfectly fits correcting single bit 
errors. This pleasant property arises because the number of error patterns that can be corrected, 2N~* — 1, 
equals the codeword length N. Codes that have 2N~* — 1 = N are known as Hamming codes, and the 
following table (Table 6.4) provides the parameters of these codes. Hamming codes are the simplest single-bit 
error correction codes, and the generator/parity check matrix formalism for channel coding and decoding 
works for them. 

Unfortunately, for such large blocks, the probability of multiple-bit errors can exceed the number of 
single-bit errors unless the channel single-bit error probability pe. is very small. Consequently, we need to 
enhance the code’s error correcting capability by adding double as well as single-bit error correction. 
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Probability of Block Error 


Signal-to-Noise Ratio (dB) 


Figure 6.23: The probability of an error occurring in transmitted K = 4 data bits equals 1 — (1 — De)* 
as (1 — pe)* equals the probability that the four bits are received without error. The upper curve displays 
how this probability of an error anywhere in the four-bit block varies with the signal-to-noise ratio. When 
a (7,4) single-bit error correcting code is used, the transmitter reduced the energy it expends during a 
single-bit transmission by 4/7, appending three extra bits for error correction. Now the probability of 
any bit in the seven-bit block being in error after error correction equals 1 — (1 pl) Tpe(1 p.)’, 
where p!, is the probability of a bit error occurring in the channel when channel coding occurs. Here 
(7pt) (1 — p.)° equals the probability of exactly on in seven bits emerging from the channel in error; The 
channel decoder corrects this type of error, and all data bits in the block are received correctly. 


Exercise 6.33 (Solution on p. 259.) 
What must the relation between N and K be for a code to correct all single- and double-bit errors 
with a “perfect fit?” 


6.30 Noisy Channel Coding Theorem* 


As the block length becomes larger, more error correction will be needed. Do codes exist that can correct 
all errors? Perhaps the crowning achievement of Claude Shannon’s creation of information theory answers 
this question. His result comes in two complementary forms: the Noisy Channel Coding Theorem and its 
converse. 


6.30.1 Statement of the Theorem 


Let E denote the efficiency of an error-correcting code: the ratio of the number of data bits to the total 
number of bits used to represent them. If the efficiency is less than the capacity of the digital channel, an 
error-correcting code exists that has the property that as the length of the code increases, the probability of 
an error occurring in the decoded block approaches zero. 

lim Pr [block error] =0 , E<C (6.58) 


N-0o 


6.30.2 Converse to the Theorem 


If E > C, the probability of an error in a decoded block must approach one regardless of the code that might 
be chosen. 
lim Pr [block error] = 1 (6.59) 


Noo 
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Figure 6.24: The capacity per transmission through a binary symmetric channel is plotted as a function 
of the digital channel’s error probability (upper) and as a function of the signal-to-noise ratio for a BPSK 
signal set (lower). 


These results mean that it is possible to transmit digital information over a noisy channel (one that in- 
troduces errors) and receive the information without error if the code is sufficiently inefficient compared 
to the channel’s characteristics. Generally, a channel’s capacity changes with the signal-to-noise ratio: As 
one increases or decreases, so does the other. The capacity measures the overall error characteristics of a 
channel—the smaller the capacity the more frequently errors occur—and an overly efficient error-correcting 
code will not build in enough error correction capability to counteract channel errors. 

This result astounded communication engineers when Shannon published it in 1948. Analog communi- 
cation always yields a noisy version of the transmitted signal; in digital communication, error correction can 
be powerful enough to correct all errors as the block length increases. The key for this capability to exist is 
that the code’s efficiency be less than the channel’s capacity. For a binary symmetric channel, the capacity 
is given by 

C=1+4 peloggpe + (1 — pe) logs (1 — pe) bits/transmission (6.60) 
Figure 6.24 (Capacity of a Channel) shows how capacity varies with error probability. For example, our (7,4) 
Hamming code has an efficiency of 0.57, and codes having the same efficiency but longer block sizes can be 
used on additive noise channels where the signal-to-noise ratio exceeds 0 dB. 


6.31 Capacity of a Channel” 


In addition to the Noisy Channel Coding Theorem and its converse, Shannon also derived the capacity for a 
bandlimited (to W Hz) additive white noise channel. For this case, the signal set is unrestricted, even to the 
point that more than one bit can be transmitted each “bit interval.” Instead of constraining channel code 
efficiency, the revised Noisy Channel Coding Theorem states that some error-correcting code exists such that 
as the block length increases, error-free transmission is possible if the source coder’s datarate, B (A) R, is 
less than capacity. 

C = Wlog, (1+ SNR) bits/s (6.61) 
This result sets the maximum datarate of the source coder’s output that can be transmitted through the 
bandlimited channel with no error. “° Shannon’s proof of his theorem was very clever, and did not indicate 
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what this code might be; it has never been found. Codes such as the Hamming code work quite well in 
practice to keep error rates low, but they remain greater than zero. Until the “magic” code is found, more 
important in communication system design is the converse. It states that if your data rate exceeds capacity, 
errors will overwhelm you no matter what channel coding you use. For this reason, capacity calculations are 
made to understand the fundamental limits on transmission rates. 


Exercise 6.34 (Solution on p. 259.) 
The first definition of capacity applies only for binary symmetric channels, and represents the 
number of bits/transmission. The second result states capacity more generally, having units of 
bits/second. How would you convert the first definition’s result into units of bits/second? 


Example 6.5 

The telephone channel has a bandwidth of 3 kHz and a signal-to-noise ratio exceeding 30 dB (at 
least they promise this much). The maximum data rate a modem can produce for this wireline 
channel and hope that errors will not become rampant is the capacity. 


C =3 x 10° log, (1 + 10°) 


(6.62) 
= 29.901 kbps 


Thus, the so-called 33 kbps modems operate right at the capacity limit. 


Note that the data rate allowed by the capacity can exceed the bandwidth when the signal-to-noise ratio 
exceeds 0 dB. Our results for BPSK and FSK indicated the bandwidth they require exceeds t What kind 
of signal sets might be used to achieve capacity? Modem signal sets send more than one bit /transmission 
using a number, one of the most popular of which is multi-level signaling. Here, we can transmit several 
bits during one transmission interval by representing bit by some signal’s amplitude. For example, two bits 


can be sent with a signal set comprised of a sinusoid with amplitudes of +A and os 


6.32 Comparison of Analog and Digital Communication” 


Analog communication systems, amplitude modulation (AM) radio being a typifying example, can inexpen- 
sively communicate a bandlimited analog signal from one location to another (point-to-point communication) 
or from one point to many (broadcast). Although it is not shown here, the coherent receiver (Figure 6.6) 
provides the largest possible signal-to-noise ratio for the demodulated message. An analysis of this receiver 
thus indicates that some residual error will always be present in an analog system’s output. 

Although analog systems are less expensive in many cases than digital ones for the same application, 
digital systems offer much more efficiency, better performance, and much greater flexibility. 


e Efficiency: The Source Coding Theorem allows quantification of just how complex a given message 
source is and allows us to exploit that complexity by source coding (compression). In analog commu- 
nication, the only parameters of interest are message bandwidth and amplitude. We cannot exploit 
signal structure to achieve a more efficient communication system. 

e Performance: Because of the Noisy Channel Coding Theorem, we have a specific criterion by which 
to formulate error-correcting codes that can bring us as close to error-free transmission as we might 
want. Even though we may send information by way of a noisy channel, digital schemes are capable 
of error-free transmission while analog ones cannot overcome channel disturbances; see this problem 
(Problem 6.15) for a comparison. 

e Flexibility: Digital communication systems can transmit real-valued discrete-time signals, which 
could be analog ones obtained by analog-to-digital conversion, and symbolic-valued ones (computer 
data, for example). Any signal that can be transmitted by analog means can be sent by digital means, 
with the only issue being the number of bits used in A/D conversion (how accurately do we need 
to represent signal amplitude). Images can be sent by analog means (commercial television), but 
better communication performance occurs when we use digital systems (HDTV). In addition to digital 
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communication’s ability to transmit a wider variety of signals than analog systems, point-to-point 
digital systems can be organized into global (and beyond as well) systems that provide efficient and 
flexible information transmission. Computer networks, explored in the next section, are what we 
call such systems today. Even analog-based networks, such as the telephone system, employ modern 
computer networking ideas rather than the purely analog systems of the past. 


Consequently, with the increased speed of digital computers, the development of increasingly efficient algo- 
rithms, and the ability to interconnect computers to form a communications infrastructure, digital commu- 
nication is now the best choice for many situations. 


6.33 Communication Networks” 


Communication networks elaborate the Fundamental Model of Communications (Figure 1.3). The model 
shown in Figure 6.25 describes point-to-point communications well, wherein the link between transmitter 
and receiver is straightforward, and they have the channel to themselves. One modern example of this 
communications mode is the modem that connects a personal computer with an information server via 
a telephone line. The key aspect, some would say flaw, of this model is that the channel is dedicated: 
Only one communications link through the channel is allowed for all time. Regardless whether we have 
a wireline or wireless channel, communication bandwidth is precious, and if it could be shared without 
significant degradation in communications performance (measured by signal-to-noise ratio for analog signal 
transmission and by bit-error probability for digital transmission) so much the better. 


Communication Network 


N 


Source 


Se eee. 


a a ee ey a a a ae ee 


Figure 6.25: The prototypical communications network—whether it be the postal service, cellular 
telephone, or the Internet—consists of nodes interconnected by links. Messages formed by the source are 
transmitted within the network by dynamic routing. Two routes are shown. The longer one would be 
used if the direct link were disabled or congested. 


The idea of a network first emerged with perhaps the oldest form of organized communication: the postal 
service. Most communication networks, even modern ones, share many of its aspects. 


A user writes a letter, serving in the communications context as the message source. 
This message is sent to the network by delivery to one of the network’s public entry points. Entry 
points in the postal case are mailboxes, post offices, or your friendly mailman or mailwoman picking 
up the letter. 

e The communications network delivers the message in the most efficient (timely) way possible, trying 
not to corrupt the message while doing so. 

e The message arrives at one of the network’s exit points, and is delivered to the recipient (what we have 
termed the message sink). 


Exercise 6.35 (Solution on p. 259.) 
Develop the network model for the telephone system, making it as analogous as possible with the 
postal service-communications network metaphor. 


What is most interesting about the network system is the ambivalence of the message source and sink about 
how the communications link is made. What they do care about is message integrity and communications 
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efficiency. Furthermore, today’s networks use heterogeneous links. Communication paths that form the 
Internet use wireline, optical fiber, and satellite communication links. 

The first electrical communications network was the telegraph. Here the network consisted of telegraph 
operators who transmitted the message efficiently using Morse code and routed the message so that it took 
the shortest possible path to its destination while taking into account internal network failures (downed lines, 
drunken operators). From today’s perspective, the fact that this nineteenth century system handled digital 
communications is astounding. Morse code, which assigned a sequence of dots and dashes to each letter of 
the alphabet, served as the source coding algorithm. The signal set consisted of a short and a long pulse. 
Rather than a matched filter, the receiver was the operator’s ear, and he wrote the message (translating 
from received bits to symbols). 


NOTE: Because of the need for a comma between dot-dash sequences to define letter (symbol) 
boundaries, the average number of bits/symbol, as described in Subtleties of Coding (Example 6.4), 
exceeded the Source Coding Theorem’s upper bound. 


Internally, communication networks do have point-to-point communication links between network nodes 
well described by the Fundamental Model of Communications. However, many messages share the com- 
munications channel between nodes using what we call time-domain multiplexing: Rather than the 
continuous communications mode implied in the Model as presented, message sequences are sent, sharing 
in time the channel’s capacity. At a grander viewpoint, the network must route messages—decide what 
nodes and links to use—based on destination information—the address—that is usually separate from the 
message information. Routing in networks is necessarily dynamic: The complete route taken by messages is 
formed as the network handles the message, with nodes relaying the message having some notion of the best 
possible path at the time of transmission. Note that no omnipotent router views the network as a whole and 
pre-determines every message’s route. Certainly in the case of the postal system dynamic routing occurs, 
and can consider issues like inoperative and overly busy links. In the telephone system, routing takes place 
when you place the call; the route is fixed once the phone starts ringing. Modern communication networks 
strive to achieve the most efficient (timely) and most reliable information delivery system possible. 


6.34 Message Routing” 


Focusing on electrical networks, most analog ones make inefficient use of communication links because truly 
dynamic routing is difficult, if not impossible, to obtain. In radio networks, such as commercial television, 
each station has a dedicated portion of the electromagnetic spectrum, and this spectrum cannot be shared 
with other stations or used in any other than the regulated way. The telephone network is more dynamic, 
but once it establishes a call the path through the network is fixed. The users of that path control its 
use, and may not make efficient use of it (long pauses while one person thinks, for example). Telephone 
network customers would be quite upset if the telephone company momentarily disconnected the path so 
that someone else could use it. This kind of connection through a network—fixed for the duration of the 
communication session—is known as a circuit-switched connection. 

During the 1960s, it was becoming clear that not only was digital communication technically superior, 
but also that the wide variety of communication modes—computer login, file transfer, and electronic mail— 
needed a different approach than point-to-point. The notion of computer networks was born then, and what 
was then called the ARPANET, now called the Internet, was born. Computer networks elaborate the basic 
network model by subdividing messages into smaller chunks called packets(Figure 6.26). The rationale for 
the network enforcing smaller transmissions was that large file transfers would consume network resources 
all along the route, and, because of the long transmission time, a communication failure might require 
retransmission of the entire file. By creating packets, each of which has its own address and is routed 
independently of others, the network can better manage congestion. The analogy is that the postal service, 
rather than sending a long letter in the envelope you provide, opens the envelope, places each page in a 
separate envelope, and using the address on your envelope, addresses each page’s envelope accordingly, and 
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Figure 6.26: Long messages, such as files, are broken into separate packets, then transmitted over com- 
puter networks. A packet, like a letter, contains the destination address, the return address (transmitter 
address), and the data. The data includes the message part and a sequence number identifying its order 
in the transmitted message. 


mails them separately. The network does need to make sure packet sequence (page numbering) is maintained, 
and the network exit point must reassemble the original message accordingly. 

Communications networks are now categorized according to whether they use packets or not. A system 
like the telephone network is said to be circuit switched: The network establishes a fixed route that lasts 
the entire duration of the message. Circuit switching has the advantage that once the route is determined, 
the users can use the capacity provided them however they like. Its main disadvantage is that the users 
may not use their capacity efficiently, clogging network links and nodes along the way. Packet-switched 
networks continuously monitor network utilization, and route messages accordingly. Thus, messages can, on 
the average, be delivered efficiently, but the network cannot guarantee a specific amount of capacity to the 
users. 


6.35 Network architectures and interconnection™ 


The network structure—its architecture (Figure 6.25)—typifies what are known as wide area net- 
works(WANs). The nodes, and users for that matter, are spread geographically over long distances. “Long” 
has no precise definition, and is intended to suggest that the communication links vary widely. The Internet 
is certainly the largest WAN, spanning the entire earth and beyond. Local area networks, LANs, employ 
a single communication link and special routing. Perhaps the best known LAN is Ethernet*®. LANs connect 
to other LANs and to wide area networks through special nodes known as gateways(Figure 6.27). In the 
Internet, a computer’s address consists of a four byte sequence, which is known as its IP address (Inter- 
net Protocol address). An example address is 128.42.4.32: each byte is separated by a period. The first 
two bytes specify the computer’s domain(here Rice University). Computers are also addressed by a more 
human-readable form: a sequence of alphabetic abbreviations representing institution, type of institution, 
and computer name. A given computer has both names (128.42.4.32 is the same as soma.rice.edu). 
Data transmission on the Internet requires the numerical form. So-called name servers translate between 
alphabetic and numerical forms, and the transmitting computer requests this translation before the message 
is sent to the network. 


6.36 Ethernet” 


Ethernet uses as its communication medium a single length of coaxial cable (Figure 6.28). This cable serves 
as the “ether,” through which all digital data travel. Electrically, computers interface to the coaxial cable 
(Figure 6.28) through a device known as a transceiver. This device is capable of monitoring the voltage 
appearing between the core conductor and the shield as well as applying a voltage to it. Conceptually it 
consists of two op-amps, one applying a voltage corresponding to a bit stream (transmitting data) and another 
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Wide-Area Network 


Figure 6.27: The gateway serves as an interface between local area networks and the Internet. The two 
shown here translate between LAN and WAN protocols; one of these also interfaces between two LANs, 
presumably because together the two LANs would be geographically too dispersed. 


Terminator Terminator 


Transceiver Transceiver 
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Figure 6.28: The Ethernet architecture consists of a single coaxial cable terminated at either end by a 
resistor having a value equal to the cable’s characteristic impedance. Computers attach to the Ethernet 
through an interface known as a transceiver because it sends as well as receives bit streams represented 
as analog voltages. 


serving as an amplifier of Ethernet voltage signals (receiving data). The signal set for Ethernet resembles 
that shown in BPSK Signal Sets, with one signal the negative of the other. Computers are attached in 
parallel, resulting in the circuit model for Ethernet shown in Figure 6.29. 


Exercise 6.36 (Solution on p. 259.) 
From the viewpoint of a transceiver’s sending op-amp, what is the load it sees and what is the 
transfer function between this output voltage and some other transceiver’s receiving circuit? Why 
should the output resistor Roy, be large? 


No one computer has more authority than any other to control when and how messages are sent. Without 
scheduling authority, you might well wonder how one computer sends to another without the (large) inter- 
ference that the other computers would produce if they transmitted at the same time. The innovation of 
Ethernet is that computers schedule themselves by a random-access method. This method relies on the 


fact 


that all packets transmitted over the coaxial cable can be received by all transceivers, regardless of 


which computer might actually be the intended recipient. In communications terminology, Ethernet directly 
supports broadcast. Each computer goes through the following steps to send a packet. 


1. 
2. 


The computer senses the voltage across the cable to determine if some other computer is transmitting. 
If another computer is transmitting, wait until the transmissions finish and go back to the first step. 
If the cable has no transmissions, begin transmitting the packet. 

If the receiver portion of the transceiver determines that no other computer is also sending a packet, 
continue transmitting the packet until completion. 
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Figure 6.29: The top circuit expresses a simplified circuit model for a transceiver. The output resistance 
Rout must be much larger than Zp so that the sum of the various transmitter voltages add to create the 
Ethernet conductor-to-shield voltage that serves as the received signal r (t) for all transceivers. In this 
case, the equivalent circuit shown in the bottom circuit applies. 


4. On the other hand, if the receiver senses interference from another computer’s transmissions, immedi- 
ately cease transmission, waiting a random amount of time to attempt the transmission again (go to 
step 1) until only one computer transmits and the others defer. The condition wherein two (or more) 
computers’ transmissions interfere with others is known as a collision. 


The reason two computers waiting to transmit may not sense the other’s transmission immediately arises 
because of the finite propagation speed of voltage signals through the coaxial cable. The longest time any 
computer must wait to determine if its transmissions do not encounter interference is 2h where L is the 
coaxial cable’s length. The maximum-length-specification for Ethernet is 1 km. Assuming a propagation 
speed of 2/3 the speed of light, this time interval is more than 10 ys. As analyzed in Problem 6.31, the 
number of these time intervals required to resolve the collision is, on the average, less than two! 


Exercise 6.37 (Solution on p. 259.) 
Why does the factor of two enter into this equation? Consider the worst-case situation of two 
transmitting computers located at the Ethernet’s ends. 


Thus, despite not having separate communication paths among the computers to coordinate their transmis- 
sions, the Ethernet random access protocol allows computers to communicate without only a slight degra- 
dation in efficiency, as measured by the time taken to resolve collisions relative to the time the Ethernet is 
used to transmit information. 

A subtle consideration in Ethernet is the minimum packet size Pin. The time required to transmit such 
packets equals Pig where C’ is the Ethernet’s capacity in bps. Ethernet now comes in two different types, 
each with individual specifications, the most distinguishing of which is capacity: 10 Mbps and 100 Mbps. If 
the minimum transmission time is such that the beginning of the packet has not propagated the full length 
of the Ethernet before the end-of-transmission, it is possible that two computers will begin transmission at 
the same time and, by the time their transmissions cease, the other’s packet will not have propagated to 
the other. In this case, computers in-between the two will sense a collision, which renders both computer’s 
transmissions senseless to them, without the two transmitting computers knowing a collision has occurred at 
all! For Ethernet to succeed, we must have the minimum packet transmission time exceed twice the voltage 


239 


Prin 
C 


propagation time: >— or 
Cc 


20 
Prin > ane (6.63) 


Thus, for the 10 Mbps Ethernet having a 1 km maximum length specification, the minimum packet size is 
200 bits. 


Exercise 6.38 (Solution on p. 259.) 
The 100 Mbps Ethernet was designed more recently than the 10 Mbps alternative. To maintain 
the same minimum packet size as the earlier, slower version, what should its length specification 
be? Why should the minimum packet size remain the same? 


6.37 Communication Protocols” 


The complexity of information transmission in a computer network—reliable transmission of bits across 
a channel, routing, and directing information to the correct destination within the destination computers 
operating system—demands an overarching concept of how to organize information delivery. No unique set of 
rules satisfies the various constraints communication channels and network organization place on information 
transmission. For example, random access issues in Ethernet are not present in wide-area networks such as 
the Internet. A protocol is a set of rules that governs how information is delivered. For example, to use 
the telephone network, the protocol is to pick up the phone, listen for a dial tone, dial a number having a 
specific number of digits, wait for the phone to ring, and say hello. In radio, the station uses amplitude or 
frequency modulation with a specific carrier frequency and transmission bandwidth, and you know to turn on 
the radio and tune in the station. In technical terms, no one protocol or set of protocols can be used for any 
communication situation. Be that as it may, communication engineers have found that a common thread 
runs through the organization of the various protocols. This grand design of information transmission 
organization runs through all modern networks today. 

What has been defined as a networking standard is a layered, hierarchical protocol organization. As 
shown in Figure 6.30, protocols are organized by function and level of detail. Segregation of information 
transmission, manipulation, and interpretation into these categories directly affects how communication 
systems are organized, and what role(s) software systems fulfill. Although not thought about in this way 
in earlier times, this organizational structure governs the way communication engineers think about all 
communication systems, from radio to the Internet. 


Exercise 6.39 (Solution on p. 259.) 
How do the various aspects of establishing and maintaining a telephone conversation fit into this 
layered protocol organization? 


We now explicitly state whether we are working in the physical layer (signal set design, for example), the 
data link layer (source and channel coding), or any other layer. IP abbreviates Internet protocol, and governs 
gateways (how information is transmitted between networks having different internal organizations). TCP 
(transmission control protocol) governs how packets are transmitted through a wide-area network such as 
the Internet. Telnet is a protocol that concerns how a person at one computer logs on to another computer 
across a network. A moderately high level protocol such as telnet, is not concerned with what data links 
(wireline or wireless) might have been used by the network or how packets are routed. Rather, it establishes 
connections between computers and directs each byte (presumed to represent a typed character) to the 
appropriate operation system component at each end. It is not concerned with what the characters mean 
or what programs the person is typing to. That aspect of information transmission is left to protocols at 
higher layers. 

Recently, an important set of protocols created the World Wide Web. These protocols exist independently 
of the Internet. The Internet insures that messages are transmitted efficiently and intact; the Internet is not 
concerned (to date) with what messages contain. HTTP (hypertext transfer protocol) frame what messages 
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signal set] Physical 


ISO Network Protocol Standard 


Figure 6.30: Protocols are organized according to the level of detail required for information transmis- 
sion. Protocols at the lower levels (shown toward the bottom) concern reliable bit transmission. Higher 
level protocols concern how bits are organized to represent information, what kind of information is de- 
fined by bit sequences, what software needs the information, and how the information is to be interpreted. 
Bodies such as the IEEE (Institute for Electronics and Electrical Engineers) and the ISO (International 
Standards Organization) define standards such as this. Despite being a standard, it does not constrain 
protocol implementation so much that innovation and competitive individuality are ruled out. 


contain and what should be done with the data. The extremely rapid development of the Web on top of 
an essentially stagnant Internet is but one example of the power of organizing how information transmission 
occurs without overly constraining the details. 


6.38 Information Communication Problems” 


Problem 6.1: Signals on Transmission Lines 

A modulated signal needs to be sent over a transmission line having a characteristic impedance of Z) = 502. 
So that the signal does not interfere with signals others may be transmitting, it must be bandpass filtered 
so that its bandwidth is 1 MHz and centered at 3.5 MHz. The filter’s gain should be one in magnitude. An 
op-amp filter (Figure 6.31) is proposed. 


(a) What is the transfer function between the input voltage and the voltage across the transmission line? 
(b) Find values for the resistors and capacitors so that design goals are met. 
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Figure 6.31 


Problem 6.2: Noise in AM Systems 

The signal s(t) emerging from an AM communication system consists of two parts: the message signal, s (t), 
and additive noise. The plot (Figure 6.32) shows the message spectrum S$(f) and noise power spectrum 
Py (f). The noise power spectrum lies completely within the signal’s band, and has a constant value there 


eae 
a 
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Figure 6.32 


(a) What is the message signal’s power? What is the signal-to-noise ratio? 

(b) Because the power in the message decreases with frequency, the signal-to-noise ratio is not constant 
within sub-bands. What is the signal-to-noise ratio in the upper half of the frequency band? 

(c) A clever ELEC 241 student suggests filtering the message before the transmitter modulates it so that 
the signal spectrum is balanced (constant) across frequency. Realizing that this filtering affects the 
message signal, the student realizes that the receiver must also compensate for the message to arrive 
intact. Draw a block diagram of this communication system. How does this system’s signal-to-noise 
ratio compare with that of the usual AM radio? 


Problem 6.3: Complementary Filters 
Complementary filters usually have “opposite” filtering characteristics (like a lowpass and a highpass) 
and have transfer functions that add to one. Mathematically, H) (f) and H2(f) are complementary if 


Ay, (f)+ He(f) =1 


We can use complementary filters to separate a signal into two parts by passing it through each filter. Each 
output can then be transmitted separately and the original signal reconstructed at the receiver. Let’s assume 


the message is bandlimited to WHz and that H; (f) = oe 
at j2n 


(a) What circuits would be used to produce the complementary filters? 
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(b) Sketch a block diagram for a communication system (transmitter and receiver) that employs comple- 
mentary signal transmission to send a message m (t). 

(c) What is the receiver’s signal-to-noise ratio? How does it compare to the standard system that sends 
the signal by simple amplitude modulation? 


Problem 6.4: Phase Modulation 
A message signal m(t) phase modulates a carrier if the transmitted signal equals 


x(t) = Asin(27f.t + dam (t)) 


where ¢g is known as the phase deviation. In this problem, the phase deviation is small. As with all analog 
modulation schemes, assume that |m (t) | < 1, the message is bandlimited to W Hz, and the carrier frequency 
fc is much larger than W. 


(a) What is the transmission bandwidth? 
(b) Find a receiver for this modulation scheme. 
(c) What is the signal-to-noise ratio of the received signal? 


HINT: Use the facts that cos (a) * 1 and sin (#) + x for small x. 


Problem 6.5: Digital Amplitude Modulation 

Two ELEC 241 students disagree about a homework problem. The issue concerns the discrete-time signal 
s(n) cos (27 fon), where the signal s(n) has no special characteristics and the modulation frequency fo is 
known. Sammy says that he can recover s(n) from its amplitude-modulated version by the same approach 
used in analog communications. Samantha says that approach won’t work. 


(a) What is the spectrum of the modulated signal? 

(b) Who is correct’? Why? 

(c) The teaching assistant does not want to take sides. He tells them that if s(n)cos(27fon) and 
s(n) sin (27fon) were both available, s(n) can be recovered. What does he have in mind? 


Problem 6.6: Anti-Jamming 

One way for someone to keep people from receiving an AM transmission is to transmit noise at the same 
carrier frequency. Thus, if the carrier frequency is f, so that the transmitted signal is Ap (1+m (t) ) sin (27 fet) 
the jammer would transmit Ajn(t) sin (27 f-t + ¢). The noise n(t) has a constant power density spectrum 


N 
over the bandwidth of the message m (t). The channel adds white noise of spectral height af 


(a) What would be the output of a traditional AM receiver tuned to the carrier frequency f.? 

(b) RU Electronics proposes to counteract jamming by using a different modulation scheme. The scheme’s 
transmitted signal has the form Ar (1 + m(t)) c(t) where c(t) is a periodic carrier signal (period #) 
having the indicated waveform (Figure 6.33). What is the spectrum of the transmitted signal with 
the proposed scheme? Assume the message bandwidth W is much less than the fundamental carrier 
frequency fe. 

(c) The jammer, unaware of the change, is transmitting with a carrier frequency of f., while the receiver 
tunes a standard AM receiver to a harmonic of the carrier frequency. What is the signal-to-noise ratio 
of the receiver tuned to the harmonic having the largest power that does not contain the jammer? 
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Figure 6.33 


Problem 6.7: Secret Communications 

A system for hiding AM transmissions has the transmitter randomly switching between two carrier fre- 
quencies f; and fg. “Random switching” means that one carrier frequency is used for some period of time, 
switches to the other for some other period of time, back to the first, etc. The receiver knows what the 
carrier frequencies are but not when carrier frequency switches occur. Consequently, the receiver must be 
designed to receive the transmissions regardless of which carrier frequency is used. Assume the message 


N 
signal has bandwidth W. The channel adds white noise of spectral height a 


(a) How different should the carrier frequencies be so that the message could be received? 
(b) What receiver would you design? 
(c) What signal-to-noise ratio for the demodulated signal does your receiver yield? 


Problem 6.8: AM Stereo 

Stereophonic radio transmits two signals simultaneously that correspond to what comes out of the left and 
right speakers of the receiving radio. While FM stereo is commonplace, AM stereo is not, but is much simpler 
to understand and analyze. An amazing aspect of AM stereo is that both signals are transmitted within the 
same bandwidth as used to transmit just one. Assume the left and right signals are bandlimited to W Hz. 


a(t) = A(1+ mj (t)) cos (27 fet) + Am, (t) sin (27 fet) 


(a) Find the Fourier transform of x(t). What is the transmission bandwidth and how does it compare 
with that of standard AM? 

(b) Let us use a coherent demodulator as the receiver, shown in Figure 6.34. Show that this receiver indeed 
works: It produces the left and right signals separately. 

(c) Assume the channel adds white noise to the transmitted signal. Find the signal-to-noise ratio of each 
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Figure 6.34 
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Problem 6.9: A Novel Communication System 

A clever system designer claims that the depicted transmitter (Figure 6.35) has, despite its complexity, 
advantages over the usual amplitude modulation system. The message signal m (t) is bandlimited to W Hz, 
and the carrier frequency (f- >> W). The channel attenuates the transmitted signal x(t) and adds white 


noise of spectral height ae 


Asin 2nfct 


m(t) x(t) 
—_ 
A cos 2nfct 
Figure 6.35 
if 
The transfer function H (f) is given by H (f) = a ; : y= 
—j, if f >0 


(a) Find an expression for the spectrum of  (t). Sketch your answer. 

(b) Show that the usual coherent receiver demodulates this signal. 

(c) Find the signal-to-noise ratio that results when this receiver is used. 

(d) Find a superior receiver (one that yields a better signal-to-noise ratio), and analyze its performance. 


Problem 6.10: Multi-Tone Digital Communication 
In a so-called multi-tone system, several bits are gathered together and transmitted simultaneously on 
different carrier frequencies during a T' second interval. For example, B bits would be transmitted according 


to 
B-1 


x(t)=AS~ bysin2n(k+1) fot , O<t<T (6.64) 
k=0 
Here, fo is the frequency offset for each bit and it is harmonically related to the bit interval T. The value of 
by is either —1 or +1. 


(a) Find a receiver for this transmission scheme. 

(b) An ELEC 241 alumni likes digital systems so much that he decides to produce a discrete-time version. 
He samples the received signal (sampling interval T,; = T/N). How should N be related to B, the 
number of simultaneously transmitted bits? 

(c) The alumni wants to find a simple form for the receiver so that his software implementation runs as 
efficiently as possible. How would you recommend he implement the receiver? 


Problem 6.11: City Radio Channels 

In addition to additive white noise, metropolitan cellular radio channels also contain multipath: the atten- 
uated signal and a delayed, further attenuated signal are received superimposed. As shown in Figure 6.36, 
multipath occurs because the buildings reflect the signal and the reflected path length between transmitter 
and receiver is longer than the direct path. 
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Figure 6.36 


(a) Assume that the length of the direct path is d meters and the reflected path is 1.5 times as long. What 
is the model for the channel, including the multipath and the additive noise? 

(b) Assume dis 1 km. Find and sketch the magnitude of the transfer function for the multipath component 
of the channel. How would you characterize this transfer function? 

(c) Would the multipath affect AM radio? If not, why not; if so, how so? Would analog cellular telephone, 
which operates at much higher carrier frequencies (800 MHz vs. 1 MHz for radio), be affected or not? 
Analog cellular telephone uses amplitude modulation to transmit voice. 

(d) How would the usual AM receiver be modified to minimize multipath effects? Express your modified 
receiver as a block diagram. 


Problem 6.12: Downlink Signal Sets 

In digital cellular telephone systems, the base station (transmitter) needs to relay different voice signals to 
several telephones at the same time. Rather than send signals at different frequencies, a clever Rice engineer 
suggests using a different signal set for each data stream. For example, for two simultaneous data streams, 
she suggests BPSK signal sets that have the depicted basic signals (Figure 6.37). 


Sy (t) So(t) 
A A 
T ' i ' 
—A —-A 
Figure 6.37 


Thus, bits are represented in data stream 1 by s; (t) and —sj (t) and in data stream 2 by s2 (t) and —s: (6), 
each of which are modulated by 900 MHz carrier. The transmitter sends the two data streams so that their 
bit intervals align. Each receiver uses a matched filter for its receiver. The requirement is that each receiver 
not receive the other’s bit stream. 


(a) What is the block diagram describing the proposed system? 

(b) What is the transmission bandwidth required by the proposed system? 

(c) Will the proposal work? Does the fact that the two data streams are transmitted in the same bandwidth 
at the same time mean that each receiver’s performance is affected? Can each bit stream be received 
without interference from the other? 
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Problem 6.13: Mixed Analog and Digital Transmission 

A signal m (t) is transmitted using amplitude modulation in the usual way. The signal has bandwidth W 
Hz, and the carrier frequency is f,. In addition to sending this analog signal, the transmitter also wants to 
send ASCII text in an auxiliary band that lies slightly above the analog transmission band. Using an 8-bit 
representation of the characters and a simple baseband BPSK signal set (the constant signal +1 corresponds 
to a 0, the constant —1 to a 1), the data signal d(t) representing the text is transmitted as the same time 
as the analog signal m(t). The transmission signal spectrum is as shown (Figure 6.38), and has a total 
bandwidth B. 


analog | digital 


ami 
fc 


Figure 6.38 


(a) Write an expression for the time-domain version of the transmitted signal in terms of m (t) and the 
digital signal d (t). 

(b) What is the maximum datarate the scheme can provide in terms of the available bandwidth? 

(c) Find a receiver that yields both the analog signal and the bit stream. 


Problem 6.14: Digital Stereo 

Just as with analog communication, it should be possible to send two signals simultaneously over a digital 
channel. Assume you have two CD-quality signals (each sampled at 44.1 kHz with 16 bits/sample). One 
suggested transmission scheme is to use a quadrature BPSK scheme. If 6) (n) and b°) (n) each represent a 
bit stream, the transmitted signal has the form 


z(t)=A>~ (0 (n) sin (27 f (t — nT)) p(t — nT) + 6 (n) cos (27 fe (t — nT)) p(t — n)) 


where p(t) is a unit-amplitude pulse having duration T and b) (n), b@ (n) equal either +1 or -1 according 
to the bit being transmitted for each signal. The channel adds white noise and attenuates the transmitted 
signal. 


(a) What value would you choose for the carrier frequency f.? 
(b) What is the transmission bandwidth? 
(c) What receiver would you design that would yield both bit streams? 


Problem 6.15: Digital and Analog Speech Communication 
Suppose we transmit speech signals over comparable digital and analog channels. We want to compare the 
resulting quality of the received signals. Assume the transmitters use the same power, and the channels 
introduce the same attenuation and additive white noise. Assume the speech signal has a 4 kHz bandwidth 
and, in the digital case, is sampled at an 8 kHz rate with eight-bit A/D conversion. Assume simple binary 
source coding and a modulated BPSK transmission scheme. 


(a) What is the transmission bandwidth of the analog (AM) and digital schemes? 

(b) Assume the speech signal’s amplitude has a magnitude less than one. What is maximum amplitude 
quantization error introduced by the A/D converter? 

(c) In the digital case, each bit in quantized speech sample is received in error with probability p, that 

depends on signal-to-noise ratio xe. However, errors in each bit have a different impact on the error in 

the reconstructed speech sample. Find the mean-squared error between the transmitted and received 

amplitude. 
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(d) In the digital case, the recovered speech signal can be considered to have two noise sources added 
to each sample’s true value: One is the A/D amplitude quantization noise and the second is due to 
channel errors. Because these are separate, the total noise power equals the sum of these two. What 
is the signal-to-noise ratio of the received speech signal as a function of pe? 

(e) Compute and plot the received signal’s signal-to-noise ratio for the two transmission schemes for a few 
values of channel signal-to-noise ratios. 

(f) Compare and evaluate these systems. 


Problem 6.16: Source Compression 
Consider the following 5-letter source. 


Letter | Probability 
a 0.5 
b 0.25 
c 0.125 
d 0.0625 
e 0.0625 
Table 6.5 


(a) Find this source’s entropy. 

(b) Show that the simple binary coding is inefficient. 

(c) Find an unequal-length codebook for this sequence that satisfies the Source Coding Theorem. Does 
your code achieve the entropy limit? 

(d) How much more efficient is this code than the simple binary code? 


Problem 6.17: Source Compression 
Consider the following 5-letter source. 


Letter | Probability 
a 0.4 
b 0.2 
Cc 0.15 
d 0.15 
e 0.1 
Table 6.6 


(a) Find this source’s entropy. 
(b) Show that the simple binary coding is inefficient. 
(c) Find the Huffman code for this source. What is its average code length? 


Problem 6.18: Speech Compression 

When we sample a signal, such as speech, we quantize the signal’s amplitude to a set of integers. For a b-bit 
converter, signal amplitudes are represented by 2° integers. Although these integers could be represented by 
a binary code for digital transmission, we should consider whether a Huffman coding would be more efficient. 


(a) Load into Matlab the segment of speech contained in y.mat. Its sampled values lie in the interval 
(-1, 1). To simulate a 3-bit converter, we use Matlab’s round function to create quantized amplitudes 
corresponding to the integers [0 1 2 3 4 5 6 7]. 
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y_quant = round(3.5*y + 3.5); 


Find the relative frequency of occurrence of quantized amplitude values. The following Matlab program 
computes the number of times each quantized value occurs. 


for n=0:7; count(n+1) = sum(y_quant == n); end; 


Find the entropy of this source. 

(b) Find the Huffman code for this source. How would you characterize this source code in words? 

(c) How many fewer bits would be used in transmitting this speech segment with your Huffman code in 
comparison to simple binary coding? 


Problem 6.19: Digital Communication 
In a digital cellular system, a signal bandlimited to 5 kHz is sampled with a two-bit A/D converter at its 
Nyquist frequency. The sample values are found to have the shown relative frequencies. 


Sample Value | Probability 
0 0.15 
1 0.35 
2 0.3 
3 0.2 
Table 6.7 


We send the bit stream consisting of Huffman-coded samples using one of the two depicted signal sets 
(Figure 6.39). 


Signal Set 1 Signal Set 2 


So(t) t t t 
A 0 F S1(t) Asolt) Ae 


Figure 6.39 


(a) What is the datarate of the compressed source? 

(b) Which choice of signal set maximizes the communication system’s performance? 

(c) With no error-correcting coding, what signal-to-noise ratio would be needed for your chosen signal set 
to guarantee that the bit error probability will not exceed 1073? If the receiver moves twice as far 
from the transmitter (relative to the distance at which the 10~% error rate was obtained), how does 
the performance change? 


Problem 6.20: Signal Compression 
Letters drawn from a four-symbol alphabet have the indicated probabilities. 
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Letter | Probability 
a 1/3 
b 1/3 
c 1/4 
d 1/12 
Table 6.8 


(a) What is the average number of bits necessary to represent this alphabet? 

(b) Using a simple binary code for this alphabet, a two-bit block of data bits naturally emerges. Find an 
error correcting code for two-bit data blocks that corrects all single-bit errors. 

(c) How would you modify your code so that the probability of the letter a being confused with the letter 
dis minimized? If so, what is your new code; if not, demonstrate that this goal cannot be achieved. 


Problem 6.21: Universal Product Code 
The Universal Product Code (UPC), often known as a bar code, labels virtually every sold good. An example 
(Figure 6.40) of a portion of the code is shown. 


Figure 6.40 


Here a sequence of black and white bars, each having width d, presents an 11-digit number (consisting of 
decimal digits) that uniquely identifies the product. In retail stores, laser scanners read this code, and after 
accessing a database of prices, enter the price into the cash register. 


(a) How many bars must be used to represent a single digit? 

(b) A complication of the laser scanning system is that the bar code must be read either forwards or 
backwards. Now how many bars are needed to represent each digit? 

(c) What is the probability that the 11-digit code is read correctly if the probability of reading a single 
bit incorrectly is pe? 

(d) How many error correcting bars would need to be present so that any single bar error occurring in the 
11-digit code can be corrected? 


Problem 6.22: Error Correcting Codes 
A code maps pairs of information bits into codewords of length-5 as follows. 


Data | Codeword 
00 00000 
01 01101 
10 10111 
11 11010 


Table 6.9 
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(a) What is this code’s efficiency? 

(b) Find the generator matrix G and parity-check matrix H for this code. 

(c) Give the decoding table for this code. How many patterns of one, two, and three errors are correctly 
decoded? 


(d) What is the block error probability (the probability of any number of errors occurring in the decoded 
codeword)? 


Problem 6.23: Digital Communication 
A digital source produces sequences of nine letters with the following probabilities. 


letter albic|{|dje|f g h i 
probability Hi 3 g § 5 i6 16 i6 i6 
Table 6.10 


a) Find a Huffman code that compresses this source. How does the resulting code compare wi e bes 
Find a Huff de that thi How does th Iting cod ith the best 
possible code? 


(b) A clever engineer proposes the following (6,3) code to correct errors after transmission through a digital 
channel. 


qj = dy C4 = (dy OD dy @ ds) 
C2 =dz C5 = (dz G ds) 


c3 =d3 cg =d, 


What is the error correction capability of this code? 
(c) The channel’s bit error probability is 1/8. What kind of code should be used to transmit data over 
this channel? 


Problem 6.24: Overly Designed Error Correction Codes 

An Aggie engineer wants not only to have codewords for his data, but also to hide the information from Rice 
engineers (no fear of the UT engineers). He decides to represent 3-bit data with 6-bit codewords in which 
none of the data bits appear explicitly. 


cq = (dy OD dz) C= (dy dy ds) 
C2 = (dz © dz) C5 = (dy © dz) 
c3 = (di @d3) cg = (di @ da G ds) 


(a) Find the generator matrix G and parity-check matrix H for this code. 
(b) Find a 3 x 6 matrix that recovers the data bits from the codeword. 
(c) What is the error correcting capability of the code? 


Problem 6.25: Error Correction? 
It is important to realize that when more transmission errors than can be corrected, error correction algo- 
rithms believe that a smaller number of errors have occurred and correct accordingly. For example, consider 
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a (7,4) Hamming code having the generator matrix 


Q 

lI 
Ee oO FE OF Oo Oo eK 
Co FF Oo OC KF OG 
Ee Fe fF CO FE CO CO 
Re ea Oe OO SE 


This code corrects all single-bit error, but if a double bit error occurs, it corrects using a single-bit error 
correction approach. 


(a) How many double-bit errors can occur in a codeword? 
(b) For each double-bit error pattern, what is the result of channel decoding? Express your result as a 
binary error sequence for the data bits. 


Problem 6.26: Selective Error Correction 

We have found that digital transmission errors occur with a probability that remains constant no matter 
how “important” the bit may be. For example, in transmitting digitized signals, errors occur as frequently 
for the most significant bit as they do for the least significant bit. Yet, the former errors have a much larger 
impact on the overall signal-to-noise ratio than the latter. Rather than applying error correction to each 
sample value, why not concentrate the error correction on the most important bits? Assume that we sample 
an 8 kHz signal with an 8-bit A/D converter. We use single-bit error correction on the most significant four 
bits and none on the least significant four. Bits are transmitted using a modulated BPSK signal set over an 
additive white noise channel. 


(a) How many error correction bits must be added to provide single-bit error correction on the most 
significant bits? 

(b) How large must the signal-to-noise ratio of the received signal be to insure reliable communication? 

(c) Assume that once error correction is applied, only the least significant 4 bits can be received in error. 
How much would the output signal-to-noise ratio improve using this error correction scheme? 


Problem 6.27: Compact Disk 

Errors occur in reading audio compact disks. Very few errors are due to noise in the compact disk player; 
most occur because of dust and scratches on the disk surface. Because scratches span several bits, a single- 
bit error is rare; several consecutive bits in error are much more common. Assume that scratch and 
dust-induced errors are four or fewer consecutive bits long. The audio CD standard requires 16-bit, 44.1 kHz 
analog-to-digital conversion of each channel of the stereo analog signal. 


(a) How many error-correction bits are required to correct scratch-induced errors for each 16-bit sample? 

(b) Rather than use a code that can correct several errors in a codeword, a clever ELEC 241 engineer 
proposes interleaving consecutive coded samples. As the cartoon (Figure 6.41) shows, the bits repre- 
senting coded samples are interspersed before they are written on the CD. The CD player de-interleaves 
the coded data, then performs error-correction. Now, evaluate this proposed scheme with respect to 
the non-interleaved one. 
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samplen 1111 __»I 
sample n+1 1010 —_» 4-way 

sample n+2 0000 ——> interleaver 
sample n+3 0101) —_> 


Figure 6.41 


Problem 6.28: Communication System Design 
RU Communication Systems has been asked to design a communication system that meets the following 
requirements. 


e The baseband message signal has a bandwidth of 10 kHz. 

e The RUCS engineers find that the entropy H of the sampled message signal depends on how many 
bits 6 are used in the A/D converter (see table below). 

e The signal is to be sent through a noisy channel having a bandwidth of 25 kHz centered at 2 MHz and 
a signal-to-noise ratio within that band of 10 dB. 

e Once received, the message signal must have a signal-to-noise ratio of at least 20 dB. 


b| H 

3 2.19 

4 | 3.25 

5 | 4.28 

6 5.35 
Table 6.13 


Can these specifications be met? Justify your answer. 


Problem 6.29: HDTV 

As HDTV (high-definition television) was being developed, the FCC restricted this digital system to use in 
the same bandwidth (6 MHz) as its analog (AM) counterpart. HDTV video is sampled on a 1035 x 1840 
raster at 30 images per second for each of the three colors. The least-acceptable picture received by television 
sets located at an analog station’s broadcast perimeter has a signal-to-noise ratio of about 10 dB. 


(a) Using signal-to-noise ratio as the criterion, how many bits per sample must be used to guarantee that 
a high-quality picture, which achieves a signal-to-noise ratio of 20 dB, can be received by any HDTV 
set within the same broadcast region? 

(b) Assuming the digital television channel has the same characteristics as an analog one, how much 
compression must HDTV systems employ? 


Problem 6.30: Digital Cellular Telephones 

In designing a digital version of a wireless telephone, you must first consider certain fundamentals. First 
of all, the quality of the received signal, as measured by the signal-to-noise ratio, must be at least as good 
as that provided by wireline telephones (30 dB) and the message bandwidth must be the same as wireline 
telephone. The signal-to-noise ratio of the allocated wireless channel, which has a 5 kHz bandwidth, measured 
100 meters from the tower is 70 dB. The desired range for a cell is 1 km. Can a digital cellphone system be 
designed according to these criteria? 


Problem 6.31: Optimal Ethernet Random Access Protocols 
Assume a population of N computers want to transmit information on a random access channel. The access 
algorithm works as follows. 
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e Before transmitting, flip a coin that has probability p of coming up heads 

e If only one of the N computer’s coins comes up heads, its transmission occurs successfully, and the 
others must wait until that transmission is complete and then resume the algorithm. 

e If none or more than one head comes up, the N computers will either remain silent (no heads) or a 
collision will occur (more than one head). This unsuccessful transmission situation will be detected 
by all computers once the signals have propagated the length of the cable, and the algorithm resumes 
(return to the beginning). 


(a) What is the optimal probability to use for flipping the coin? In other words, what should p be to 
maximize the probability that exactly one computer transmits? 

(b) What is the probability of one computer transmitting when this optimal value of p is used as the 
number of computers grows to infinity? 

(c) Using this optimal probability, what is the average number of coin flips that will be necessary to resolve 
the access so that one computer successfully transmits? 

(d) Evaluate this algorithm. Is it realistic? Is it efficient? 


Problem 6.32: Repeaters 

Because signals attenuate with distance from the transmitter, repeaters are frequently employed for both 
analog and digital communication. For example, let’s assume that the transmitter and receiver are D m 
apart, and a repeater is positioned halfway between them (Figure 6.42). What the repeater does is amplify 
its received signal to exactly cancel the attenuation encountered along the first leg and to re-transmit the 
signal to the ultimate receiver. However, the signal the repeater receives contains white noise as well as the 
transmitted signal. The receiver experiences the same amount of white noise as the repeater. 


transmitter repeater receiver 
SS > > 
~« D/2 »><« D/2 , a 
~< D 7 a 
Figure 6.42 


(a) What is the block diagram for this system? 

(b) For an amplitude-modulation communication system, what is the signal-to-noise ratio of the demodu- 
lated signal at the receiver? Is this better or worse than the signal-to-noise ratio when no repeater is 
present? 

(c) For digital communication, we must consider the system’s capacity. Is the capacity larger with the 
repeater system than without it? If so, when; if not, why not? 


Problem 6.33: Designing a Speech Communication System 

We want to examine both analog and digital communication alternatives for a dedicated speech transmission 
system. Assume the speech signal has a 5 kHz bandwidth. The wireless link between transmitter and receiver 
is such that 200 watts of power can be received at a pre-assigned carrier frequency. We have some latitude in 
choosing the transmission bandwidth, but the noise power added by the channel increases with bandwidth 
with a proportionality constant of 0.1 watt/kHz. 


(a) Design an analog system for sending speech under this scenario. What is the received signal-to-noise 
ratio under these design constraints? 

(b) How many bits must be used in the A/D converter to achieve the same signal-to-noise ratio? 

(c) Is the bandwidth required by the digital channel to send the samples without error greater or smaller 
than the analog bandwidth? 
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Problem 6.34: Digital vs. Analog 

You are the Chairman/Chairwoman of the FCC. The frequency band 3 MHz to 3.5 MHz has been allocated 
for a new “high-quality” AM band. Each station licensed for this band will transmit signals having a 
bandwidth of 10 kHz, twice the message bandwidth of what current stations can send. 


(a) How many stations can be allocated to this band and with what carrier frequencies? 

(b) Looking ahead, conversion to digital transmission is not far in the future. The characteristics of the 
new digital radio system need to be established and you are the boss! Detail the characteristics of the 
analog-to-digital converter that must be used to prevent aliasing and ensure a signal-to-noise ratio of 
25 dB. 

(c) Without employing compression, how many digital radio stations could be allocated to the band if 
each station used BPSK modulation? Evaluate this design approach. 
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Solutions to Exercises in Chapter 6 


Solution to Exercise 6.1 (p. 200) 
In both cases, the answer depends less on se than on material properties. For coaxial cable, c = 


cosh + ( (#) 


1 

For twisted pair, c = : 
Vlad ‘ faz + cosh7* (4) 
Solution to Exercise 6.2 (p. 200) 
You can find these frequencies from the spectrum allocation chart (Section 7.3). Light in the middle of the 
visible band has a wavelength of about 600 nm, which corresponds to a frequency of 5 x 10'4Hz. Cable 
television transmits within the same frequency band as broadcast television (about 200 MHz or 2 x 10°Hz). 
Thus, the visible electromagnetic frequencies are over six orders of magnitude higher! 
Solution to Exercise 6.3 (p. 201) 
As shown previously (6.11), voltages and currents in a wireline channel, which is modeled as a transmission 
line having resistance, capacitance and inductance, decay exponentially with distance. The inverse-square law 
governs free-space propagation because such propagation is lossless, with the inverse-square law a consequence 
of the conservation of power. The exponential decay of wireline channels occurs because they have losses 
and some filtering. 
Solution to Exercise 6.4 (p. 202) 


Figure 6.43 


Use the Pythagorean Theorem, (h + By = R?4d?, where h is the antenna height, d is the distance from 
the top of the earth to a tangency point with the earth’s surface, and R the earth’s radius. The line-of-sight 
distance between two earth-based antennae equals 


duos = 2h R + hy? + \/2haR + hq? (6.66) 


As the earth’s radius is much larger than the antenna height, we have to a good approximation that dros = 
V2hiR+4+ V2h2R. If one antenna is at ground elevation, say hz = 0, the other antenna’s range is /2h,R. 
Solution to Exercise 6.5 (p. 202) 

As frequency decreases, wavelength increases and can approach the distance between the earth’s surface and 
the ionosphere. Assuming a distance between the two of 80 km, the relation Af = c gives a corresponding 
frequency of 3.75 kHz. Such low carrier frequencies would be limited to low bandwidth analog communication 
and to low datarate digital communications. The US Navy did use such a communication scheme to reach 
all of its submarines at once. 

Solution to Exercise 6.6 (p. 203) 

Transmission to the satellite, known as the uplink, encounters inverse-square law power losses. Reflecting 
off the ionosphere not only encounters the same loss, but twice. Reflection is the same as transmitting 
exactly what arrives, which means that the total loss is the product of the uplink and downlink losses. The 
geosynchronous orbit lies at an altitude of 35700 km. The ionosphere begins at an altitude of about 50 km. 
The amplitude loss in the satellite case is proportional to 2.8 x 1078; for Marconi, it was proportional to 
4.4 x 107!°. Marconi was very lucky. 

Solution to Exercise 6.7 (p. 204) 

If the interferer’s spectrum does not overlap that of our communications channel—the interferer is out-of- 
band—we need only use a bandpass filter that selects our transmission band and removes other portions of 
the spectrum. 
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Solution to Exercise 6.8 (p. 205) 

The additive-noise channel is not linear because it does not have the zero-input-zero-output property (even 
though we might transmit nothing, the receiver’s input consists of noise). 

Solution to Exercise 6.9 (p. 207) 

The signal-related portion of the transmitted spectrum is given by X (f) = $M (f— fe) + $M (f+ fe). 
Multiplying at the receiver by the carrier shifts this spectrum to +f. and to —f., and scales the result by 
half. 


SX (f— fe) + 5X 0 + fe) = 5 (M (f —2fe) + M(f)) + 5 (MF +2f-) + MCN) ne 
= FM (f ~2fe) + 5M (f)+ 2M Uf + 2fe) 


The signal components centered at twice the carrier frequency are removed by the lowpass filter, while the 
baseband signal M (f) emerges. 

Solution to Exercise 6.10 (p. 208) 

The key here is that the two spectra M(f— f.), M(f+f-) do not overlap because we have assumed 
that the carrier frequency f, is much greater than the signal’s highest frequency. Consequently, the term 
M (f — f.)M(f +f.) normally obtained in computing the magnitude-squared equals zero. 

Solution to Exercise 6.11 (p. 209) 

Separation is 2W. Commercial AM signal bandwidth is 5kHz. Speech is well contained in this bandwidth, 
much better than in the telephone! 

Solution to Exercise 6.12 (p. 210) 

2 (t) = O_o. (Soin) (¢— nT). 

Solution to Exercise 6.13 (p. 211) 

k=4. 

Solution to Exercise 6.14 (p. 211) 


Lo ((-1 Avr (t — nT) sin ) 


Solution to Exercise 6.15 (p. 212) 
The harmonic distortion is 10%. 
Solution to Exercise 6.16 (p. 212) 
Twice the baseband bandwidth because both positive and negative frequencies are shifted to the carrier by 
the modulation: 3R. 
Solution to Exercise 6.17 (p. 214) 
In BPSK, the signals are negatives of each other: s,(t) = —s9(t). Consequently, the output of each 
multiplier-integrator combination is the negative of the other. Choosing the largest therefore amounts to 
choosing which one is positive. We only need to calculate one of these. If it is positive, we are done. If it is 
negative, we choose the other signal. 
Solution to Exercise 6.18 (p. 214) 

2 


The matched filter outputs are + 5 because the sinusoid has less power than a pulse having the same 


amplitude. 

Solution to Exercise 6.19 (p. 216) 

The noise-free integrator outputs differ by aA?T, the factor of two smaller value than in the baseband case 
arising because the sinusoidal signals have less energy for the same amplitude. Stated in terms of Ep, the 
difference equals 2a Ey just as in the baseband case. 

Solution to Exercise 6.20 (p. 217) 


ak» 


The noise-free integrator output difference now equals a.A?T = . The noise power remains the same as 


°F 
in the BPSK case, which from the probability of error equation (6.46) yields p. = Q ( | on ) ‘ 
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Solution to Exercise 6.21 (p. 218) 


K 
that this is the maximum-entropy probability assignment, we must explicitly take into account that prob- 
abilities sum to one. Focus on a particular symbol, say the first. Pr [ao] appears twice in the entropy for- 
mula: the terms Pr [ao] log, (Pr[a9]) and (1 — Pr [ao] + --- + Pr [ax_2]) logy (1 — Pr [ap] +--+ + Pr [ax_2]). 
The derivative with respect to this probability (and all the others) must be zero. The derivative equals 
logs (Pr [ao]) — logy (1 — Pr [ao] +--+ + Pr[ax_—2]), and all other derivatives have the same form (just substi- 
tute your letter’s index). Thus, each probability must equal the others, and we are done. For the minimum 
entropy answer, one term is llog.l = 0, and the others are Olog,0, which we define to be zero also. The 
minimum value of entropy is zero. 

Solution to Exercise 6.22 (p. 221) 

The Huffman coding tree for the second set of probabilities is identical to that for the first (Figure 6.18). 


1 1 1 
Equally likely symbols each have a probability of Ke Thus, H(A) =—)>°, x 082 ( ) = log, Kk. To prove 


The average code length is 41 + 42+ 43 a3 = 1.75 bits. The entropy calculation is straightforward: 
oe line PLN ae tigi a Lite aid 1 
H (A) = —(Slog ($) + flog ($=) + glog (=) + sglog (4)), which equals 1.68 bits. 


Solution to Exercise 6.23 (p. 222) 
1 

B(A)R 
Solution to Exercise 6.24 (p. 222) 
Because no codeword begins with another’s codeword, the first codeword encountered in a bit stream must 
be the right one. Note that we must start at the beginning of the bit stream; jumping into the middle does 
not guarantee perfect decoding. The end of one codeword and the beginning of another could be a codeword, 
and we would get lost. 
Solution to Exercise 6.25 (p. 222) 
Consider the bitstream ...0110111... taken from the bitstream 0]10]110|110]111].... We would decode the 
initial part incorrectly, then would synchronize. If we had a fixed-length code (say 00,01,10,11), the situation 
is much worse. Jumping into the middle leads to no synchronization at all! 
Solution to Exercise 6.26 (p. 224) 
This question is equivalent to 3p. (1 — pe) +pe? < 1 or 2p-?+(—3) pe +1 > 0. Because this is an upward-going 
parabola, we need only check where its roots are. Using the quadratic formula, we find that they are located 
at 5 and 1. Consequently in the range 0 < pe < $ the error rate produced by coding is smaller. 
Solution to Exercise 6.27 (p. 225) 
With no coding, the average bit-error probability p,. is given by the probability of error equation (6.47): 


2a? FE 
De =Q ( a ) . With a threefold repetition code, the bit-error probability is given by 3p, (1- p,) +01", 


2a? Ey 
3No 
because of the energy reduction is not compensated by the repetition coding. 


where p!, = Q . Plotting this reveals that the increase in bit-error probability out of the channel 
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Error Probability with and without (3,1) Repetition Coding 
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Figure 6.44 


Solution to Exercise 6.28 (p. 226) 

In binary arithmetic (see Table 6.2), adding 0 to a binary value results in that binary value while adding 1 
results in the opposite binary value. 

Solution to Exercise 6.29 (p. 228) 


dmin = 2n+1 


Solution to Exercise 6.30 (p. 229) 
When we multiply the parity-check matrix times any codeword equal to a column of G, the result consists of 
the sum of an entry from the lower portion of G and itself that, by the laws of binary arithmetic, is always 
zero. 

Because the code is linear—sum of any two codewords is a codeword—we can generate all codewords as 
sums of columns of G. Since multiplying by 4 is also linear, He = 0. 
Solution to Exercise 6.31 (p. 229) 
In binary arithmetic, adding 0 to a binary value results in that binary value while adding 1 results in the 
opposite binary value. 
Solution to Exercise 6.32 (p. 230) 
The probability of a single-bit error in a length-N block is Np.(1 — De)" and a triple-bit error has prob- 


ability iC per(1— ny For the first to be greater than the second, we must have 
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For N =7, pe < 0.31. 

Solution to Exercise 6.33 (p. 230) 

In a length-N block, N single-bit and NW) double-bit errors can occur. The number of non-zero vectors 
resulting from H¢ must equal or exceed the sum of these two numbers. 


N(N-1) ware ae N?4+N+4+2 


ONS Sse Ny 
>N+ 5 > ; 


(6.68) 
The first two solutions that attain equality are (5,1) and (90,78) codes. However, no perfect code exists 
other than the single-bit error correcting Hamming code. (Perfect codes satisfy relations like (6.68) with 
equality. ) 

Solution to Exercise 6.34 (p. 233) 

To convert to bits/second, we divide the capacity stated in bits/transmission by the bit interval duration T. 
Solution to Exercise 6.35 (p. 234) 

The network entry point is the telephone handset, which connects you to the nearest station. Dialing the 
telephone number informs the network of who will be the message recipient. The telephone system forms an 
electrical circuit between your handset and your friend’s handset. Your friend receives the message via the 
same device—the handset—that served as the network entry point. 


Solution to Exercise 6.36 (p. 237) 
Rout 


The transmitting op-amp sees a load given by Rout + Zo || , where N is the number of transceivers 


other than this one attached to the coaxial cable. The transfer function to some other transceiver’s receiver 
circuit is Rou, divided by this load. 

Solution to Exercise 6.37 (p. 238) 

The worst-case situation occurs when one computer begins to transmit just before the other’s packet arrives. 
Transmitters must sense a collision before packet transmission ends. The time taken for one computer’s 
packet to travel the Ethernet’s length and for the other computer’s transmission to arrive equals the round- 
trip, not one-way, propagation time. 

Solution to Exercise 6.38 (p. 239) 

The cable must be a factor of ten shorter: It cannot exceed 100 m. Different minimum packet sizes means 
different packet formats, making connecting old and new systems together more complex than need be. 
Solution to Exercise 6.39 (p. 239) 

When you pick up the telephone, you initiate a dialog with your network interface by dialing the number. 
The network looks up where the destination corresponding to that number is located, and routes the call 
accordingly. The route remains fixed as long as the call persists. What you say amounts to high-level 
protocol while establishing the connection and maintaining it corresponds to low-level protocol. 
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Chapter 7 


Appendix 


7.1 Decibels’ 


The decibel scale expresses amplitudes and power values logarithmically. The definitions for these differ, 
but are consistent with each other. 


power (s, in decibels) = 10 log, (es 


power (50) 
amplitude (s) ) “ 


amplitude (s, in decibels) = 20 logy, cea (6) 


Here power (so) and amplitude (so) represent a reference power and amplitude, respectively. Quantifying 
power or amplitude in decibels essentially means that we are comparing quantities to a standard or that we 
want to express how they changed. You will hear statements like “The signal went down by 3 dB” and “The 
filter’s gain in the stopband is —60” (Decibels is abbreviated dB.). 


Exercise 7.1 (Solution on p. 265.) 
The prefix “deci” implies a tenth; a decibel is a tenth of a Bel. Who is this measure named for? 


The consistency of these two definitions arises because power is proportional to the square of amplitude: 
(power (s) amplitude? (s)) (7.2) 
Plugging this expression into the definition for decibels, we find that 


10 log ( power (s) ) = 10 log ( amplitude? (s) ) 
10 \ power (89) *° \ amplitude? (0) 


amplitude (s) 
amplitude (s9) 


(7.3) 
= 20 logig ( 


Because of this consistency, stating relative change in terms of decibels is unambiguous. A factor 
of 10 increase in amplitude corresponds to a 20 dB increase in both amplitude and power! 

The accompanying table provides “nice” decibel values. Converting decibel values back and forth is fun, 
and tests your ability to think of decibel values as sums and/or differences of the well-known values and of 
ratios as products and/or quotients. This conversion rests on the logarithmic nature of the decibel scale. 
For example, to find the decibel value for /2, we halve the decibel value for 2; 26 dB equals 10 + 10 + 6 dB 
that corresponds to a ratio of 10 x 10 x 4 = 400. Decibel quantities add; ratio values multiply. 

One reason decibels are used so much is the frequency-domain input-output relation for linear systems: 
Y (f) =X (f) H (f). Because the transfer function multiplies the input signal’s spectrum, to find the output 


1This content is available online at http://cnx.org/content/m0082/2.16/. 
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Power Ratio | dB 


0.1 =10 
1 0 
J2 1.5 
2 3 
V10 5 
4 6 
5 7 
8 9 
10 10 


Figure 7.1: Common values for the decibel. The decibel values for all but the powers of ten are 
approximate, but are accurate to a decimal place. 


amplitude at a given frequency we simply add the filter’s gain in decibels (relative to a reference of one) to the 
input amplitude at that frequency. This calculation is one reason that we plot transfer function magnitude 
on a logarithmic vertical scale expressed in decibels. 


7.2 Permutations and Combinations’ 


The lottery “game” consists of picking & numbers from a pool of n. For example, you select 6 numbers out 
of 60. To win, the order in which you pick the numbers doesn’t matter; you only have to choose the right set 
of 6 numbers. The chances of winning equal the number of different length-k sequences that can be chosen. 
A related, but different, problem is selecting the batting lineup for a baseball team. Now the order matters, 
and many more choices are possible than when order does not matter. 

Answering such questions occurs in many applications beyond games. In digital communications, for 
example, you might ask how many possible double-bit errors can occur in a codeword. Numbering the bit 
positions from 1 to N, the answer is the same as the lottery problem with k = 6. Solving these kind of 
problems amounts to understanding permutations- the number of ways of choosing things when order 
matters as in baseball lineups - and combinations- the number of ways of choosing things when order does 
not matter as in lotteries and bit errors. 

Calculating permutations is the easiest. If we are to pick k numbers from a pool of n, we have n choices 
for the first one. For the second choice, we have n — 1. The number of length-two ordered sequences is 
therefore be n(n — 1). Continuing to choose until we make k choices means the number of permutations 

! 


(n aT 


is n(n—1)(n—2)---(n—k+1). This result can be written in terms of factorials as with n! = 


n(n —1)(n—2)---1. For mathematical convenience, we define 0! = 1. 

When order does not matter, the number of combinations equals the number of permutations divided by 
the number of orderings. The number of ways a pool of k things can be ordered equals k!. Thus, once we 
choose the nine starters for our baseball game, we have 9! = 362,880 different lineups! The symbol for the 

n! 

(n—k)!k! 
Exercise 7.2 (Solution on p. 265.) 
What are the chances of winning the lottery? Assume you pick 6 numbers from the numbers 1-60. 


combination of k things drawn from a pool of n is (1) and equals 


Combinatorials occur in interesting places. For example, Newton derived that the n-th power of a sum 


obeyed the formula (a + y)” = (5) xu” + @ gly + (5) ty? peep a y”. 


?This content is available online at http: //cnx.org/content/m10262/2.13/. 
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Exercise 7.3 (Solution on p. 265.) 
What does the sum of binomial coefficients equal? 


k=0 


A related problem is calculating the probability that any two bits are in error in a length-n codeword when 
p is the probability of any bit being in error. The probability of any particular two-bit error sequence is 
p(ii- ey The probability of a two-bit error occurring anywhere equals this probability times the number 


n = 
of combinations: 5 p*(1—p)” ?. Note that the probability that zero or one or two, etc. errors occurring 


must be one; in other words, something must happen to the codeword! That means that we must have 


& (l—p)" + (7) oa — py i+ (5) ra BS a cs ("Jor = 


Can you prove this? 


7.3 Frequency Allocations’ 


To prevent radio stations from transmitting signals “on top of each other,” the United States and other 
national governments in the 1930s began regulating the carrier frequencies and power outputs stations could 
use. With increased use of the radio spectrum for both public and private use, this regulation has become 
increasingly important. This is the so-called Frequency Allocation Chart, which shows what kinds of 
broadcasting can occur in which frequency bands. Detailed radio carrier frequency assignments are much 
too detailed to present here. 


3This content is available online at http: //cnx.org/content/m0083/2.11/. 
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Solutions to Exercises in Chapter 7 


Solution to Exercise 7.1 (p. 261) 
Alexander Graham Bell. He developed it because we seem to perceive physical quantities like loudness and 
brightness logarithmically. In other words, percentage, not absolute differences, matter to us. We use 
decibels today because common values are small integers. If we used Bels, they would be decimal fractions, 
which aren’t as elegant. 
Solution to Exercise 7.2 (p. 262) 

60 60! 
(S) = Enel = 50, 063, 860. 
Solution to Exercise 7.3 (p. 263) 
Because of Newton’s binomial theorem, the sum equals (1 + 1)” = 2”. 
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